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standardizing approaches for development” staffs. In a0 a 
response to the challenges 6f these emerging models’ © Reg ~~ 4 
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integrated information management systems Which incorporate 
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SECTION TI. 7 
_. INTRODUCTION ~~. 7s 
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"PURPOSE OF THIS DOCUMENT oe, 2 


a 


. . . - Ce = : 
This document fs intended to provide’ the reader with a. “fm 
.eClear description.of a model “for Computer Aided Training : 
System Development ‘and Management (CATSDM) -and, is designed. 
to, serve as a functional specification base ‘from which more 
detailed design and development activities could proceed, *. 
- = - a . 3 


4 at : 
"=~ In previous documents prepared as part of the Computer 
- Aided’ Trajining System Development and Managément (CATSDM) to 
ISD (Instructional Systems Development) project, a series of - 
analyses were conducted to determine the characteristics ¢f 
‘an idealized computer hardware/software system which could 
support the Instfuctional’ System Development (ISD) process. 
Analyses were also conducted to prioritize support needs for 
the various technical ISDuprocesses. Finally, existing . 


© relevant hardware/software systems were examined to 


“determine, their. general capabilities and potentials. in 
relationship. to the scene ated requirements. 


oe . | 1 : 
~. Using’ the analyses outlined above,,a set of functional - 


+ specifications has been prepared describing a system of 


. computer programs.to assist in Successfully and reliably 


» 


carrying out the ISD process. These-programs are designed 


to lend structure to the activities of persons involved in 


NAVTRAEQUIPCEN 77-C-0018-1 sgt 


ISD, as well as.to provide a wide range of support, :.- 5 


functions, including: needed data collection and’ manipulation  , 
capabilities for, all. phases of the process. . a 


OVERVIEW OF -THE CONTENTS. , - : 
|The remainder*of this document will deal’ in detail with: 
-the components of CATSDM, conception and development of _ : 

in light of the analysis of éxisting systems, including Navy 


Training device number 11869, referred to in his document 
s,the Versatile Training System (VTS). This will not enly_ 


-Y . those. peor the & and recommendations and conclusions made 


“provide the reader with a feel. for what -CATSDM is and-does, 


.. but also with how.the benefits of CATSDM can be realized and 
integrated with what is already available through VTS2 


% . ra 

_.The section on "Description of Method. inmediatety/ 

; foliowing this section will describe the cer -al procedures 
' fodlowed in developing this d&ta item. It w..l1: outline how 


; ~ 
} a 


<* « - a 
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Section II wert contain three subsections which: ‘will 1D  e 
- overview the recommended system of ‘programs, 2) describe in ( 
detail program capabilities (both standard across the whole 
CATSDM systemiand specific “to individual subprograms) and, 

3) GescE Re the data bases created _and. used by the programs. 

Each .of the détailed decor ioe ons of’ program 7 

Capabilities will be «accompanied, by a table isting what ° 
types of information will be entered into each data base, - 
and specifying show it was entered. A large portion of the . 
, Capabilities available for One program will also be ~ 
available in other programs. Therefore, instead: of ae 
describing each capabilify over and over asa partisaf each 
program, a single set of descriptions of these’ common . -. 
; Capabriities will be presented separate from sie ivicuel 

program descriptions. By describing the..standard . *. « 

capabilities in detail once, it will then be Faasonabie: to oo 
merely reference Shem in the descriptions of each of. the 


. mabe asent programs. a a ce ~ — 
‘The description of the ‘data bases included in the third) f 
Subsection will include a-series of tables ‘outlining the. oe 4 
projected contents of each of the data bases. For each ‘ew FF 4K 
, piece of data in each data fbase, the program used to enter " . o ¥ 
that data will be identified, ane the .programs which: access _ Bis . 
that data will be listed. ~ 2 xB 4 » ; 4 


Analysis ef the VTS in light of CATSDM recommendations ; 
will be ‘provided in Section ITI¥f. -The ‘purpose of this. Wye me 2B P 
section is to evaluate the VIS.to determine h& its : ; 
capabilities relate to requirements of the pr sed GATS DM 
system. - The analysis will outline where the VTS addtess the - . 
CATSDM need as specified, ‘and where it takes an ee 
but acceptable approach to accomplishing each specific “ 
CATSDM task.° “It will also identify where -the ¥TS does not. i 
now provide the’? tTequired Gapabiiteies in any cuales = 
. 7 T . 

Section LV will. present recommended plan for Haw to. go Pog : 
about the dévelopment and implementation of the proposed / », Ee == 
CATSDM_ syst __It will specifically address how,the VTS 
could be systématically. ex panded. and médified to meet the 
y>CATSDM requirements. The plan will ‘attempt to organize 
deveYopment ‘tasks in-such a way that individual programs and 
subsystems can be developed one at a time and in(/Such a Way 
that each/will provide a useful product in and. o itself. “ve 
In this way many-of the benefits of CATSDM car be realized 
very early in the ‘development effort, without having to wait. - 
for the ‘whole. system to be completed. The section will also a 


‘present development tasks, phasing, and_- staffing. ae * 


- 


ANI - oe - Ms ; ie . eae . ; _ ae ( 
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. “Seetion Vv of the report will summarize all conclusions: 
and recommendations. ‘©. oS - 


ra a° 
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a DESCRIPTION OF METHODS 
= bs This, section will describe, the general’ procedurés which 
‘were fallowed-in developing this data item. It -will outline 
how the. VTS was Studied, analyzed, and compared to CATSDM. 

: ge This data item was prepared from several major data 

ss Fe aa ee The first obviously waS ‘the preliminary work done 
(on Previous data. items ‘and their extension into the | 

ription of-programs, data bases, and so on required--for 


ry 


CATSDM. , . 
“ss. Inform tion on the Versatile Training System (VTS) was - 
i Gathered from two erage sources. The first was a set of: the 

' best-existing writtén documentation materials available on 
7 the VTS. , . S E —, A : : 


™~. . . 


Specifically these are: ° e. . = ‘ 
- , VT$ & CATSDM - Outline of CATSDM VTS Overview CATSDM ~ 
fe Explanations , 4 a ae, g 


. "WIS = Vol. I / = FRAMP User's Manoal Nor 1% 

VTS - Vol.-II_ - FRAMP iserts Manual’ No, 2 

VTS -- Vol. III~ = PRS Aircrew Personnel User's Manual 
_ - VTS = Vol. Iv. _ © FRS Aircrew Personpel User's Mariual. ® 


aan — VTS - Vol. V.* - Subsystem Specs for. Aircrew and 
9 ? ' Enlisted Men —_— . vo owe. 
- VTS Functional Descfiptioh for 
— Naval Aviation ‘Activities | ; 
. * The general approach taken tg utilize this documentation was 
". to overview all. the written.materials to orient the | * 
. contractor staff to systems Capabilities and the r _ 
/ documentation formats and layouts used. Then, utilizing the 
| description of programs and 5 andard ,capabilitdes implied by 
sCATSDM, the documentation was searched specifically for- VTS 
- implications to each section of the CATSDM model on a a 
program-by program and capability by capability basis. me 


Finalidy, in those areas where the written documentation 


was inadequate (the documentation for VTS is not yet-. ; 
-complete) follow-up: phone calls and additional visits were 
made. to 'NWC China Lake, where questions were clarified as 1, 


5 . . 
‘ ri og 
: . a aN: 
* - — ‘ 2 : - . c 
. ” . 7 ‘ . , ee PNY 
3 ti : : . 


x 


i 


eaten - oe on is —— 
RIC ‘ 7 ‘ . E 0 


E F : i : . 7 > - 
Pru Tx Provided by ER ae ‘ , a 
. 4 ee . F < o. - x 2 a ; - 
_ ae . Pee Pad ~~ I - in 


\ 


( So . : : aid ‘ 
- NAVTRAEQUIPCEN 77-C- 0018- 1: i 


+ 


‘much as possible. In” those ‘Gases where GPor nae loa was... 


unavailable or ingomplete, assumptions were made-in the 
anaysis of VTS/CAITS DM match ups and were clearly. identif--d 
as assumptions in this report. ; 


- 


The CATSDM funetional specifications themselves were ‘the 
joint product ofa group of.contractor personnel with. 
extensive experience’ in large scale ISD Cincluding: NB we Ts 
Army ,« Marines, and Air Force projects); in.writing’. 
specifications for computér based training and training. 


“support systems, and in t*~ operational development and use 


-of such systems.. In one ceras the degree of detail 


provided in the descript:... .amd definition of the CATSDM. 
programs. and their. relationship to each other (See Section 
II) may suggest that this document hight serve as more than 
a functional specification. That Zs. not the. caser “ 
Systems—subsystem specification, databa se speci Lfications, 
and program specifications all require more de bail.than is 
given or intended here. They depend extensively uppn the 


_hardware/software environment ‘selected for a 


“constraining. - The relationships ampiied Suggest functional, 


The level of detail peovided in this repsor- is intended 
to be helpfu® and. suggestive and not limiting -- é 


not St ructual definitions. | 
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_ "SECTION II 
» DESCRIPTION. oF |PROGRaMS. - 
OVERVIEW OF THE SYSTEM OF PROGRAMS | 


2 


In as much as the ISD process is an attempt at a 


“Systematic solution to complex training problems,\ CATSDM is 


designed. to provide Systematic Structuring. of the activities 
of persons, resources, and data involved in all phases of 
the process. »Furthermore, CATSDM provides a system of 
computer programs which assists in the creation, ; 
manipulation, and updating of data bases .critical to the ISD 
process. This section presents .an overview of those — 
programs, their interrelationships, and their associated 


.data bases. It should be made clear here that these - 


"programs" are really complex sub-systems in their own 
right. ie i . - , P ae a : my 


Support is provided by CATSDM to both main line ISD 
activities clearly répresented in the five ISD phases 
(analysis, design, development, implementation, and’ . 
evaluation), and related peripheral ‘activities (trainer 
requirements/procurement, tactics package development> and 


-contr’acting concerns). The programs available’ through 


CATSDM are’ bes ‘understood when related to these groups of 


activities. In ‘order to view Program interrelationships 


more eastly’; we suggest following the flow diagram in Figure 


1 as each of the programs is discussed. : 


ay 
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‘Analysis Phase Programs: and Data Bases , e.. 4 < 


The, Problem AnalySis Program (PAP).-assists in the first _ 


'. step of the analysis phase by helping to. specify and 


. instructional program. In so doing, @ problem analysis data . 


~ 


- retyping, etc. 


interpret the kinds of data collected, and helping to © . 


determine the need for.developing or revising an. . 


base accessed by subsequent -analysis phase programs is 


‘created. This data.base also contributes to,the’Master Plan 


Frogram (MPA) ahd the Procurement Package Program (PPP), Pi 


both of which are described. later-in this overview. XY 
. The Task Listing Program (TLP) ‘assists in the creation 

of a complete set of tasks performed in the job\or jobs 

being analyzed. In addition, the. tasks are structured and © 

numbered so superordinate/subordinate relationships are ~ 

clearly shown and can be easily manipulated (inserted, 

deleted, revised) without cumbersome manual’ renumbering,. 
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‘ Batalie® o- “task fist Soucicoscat a ae ree of. 


* student. é@ytry level is made uSing the Student. Entry-Level’ 


a 


-- Program. (€SELP).-. This establishes: ‘a student. ebtry ievel: dake 


+ 


_ base and , means *for’ interpreting: that data." g < ie -¢ eS * 


: eee: ° 
x mee - cam 


ee dea eg: level. data and "task listing: ‘data bases” bre 


ei cos ‘by -the .Task Validation. ‘and “Sélectinpn; Program... 


(TWSP),- ‘to -assist. in verifying the task .list for. 


completeness’ and accuracy’ and. in -Selecting. tasks. for shich. 


' training. 3S required. AS a result >, the bask - list data~base . 


~/is updated as necessary. and ‘tasks requiring. training ‘are 


earmar ked'.: fe en : 4 ; ore i Be 


: =, : aa 
a “the. Ob jéetives Hierarchy: ‘Program (COHP) . then assists” 2 


vinstructional. developers in “establishing a. hierarchy of 
learning objectives based on the task list.” The resulting 
objectives hierarchy data base shows relationships among. 
objectives eee pteet oe Suggests the order in which they 


greundaries: for activities in. the Isb eesten phase. 


shouldbe learned.i This-<data base also provides the 2 


. 
.# 
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As ebgeee due hierarchies are created, one ‘additional . 
data base in the analysis phase is ‘established through the ~- 


Existing Materials Evaluation. Program (EMEP). ° This helps 


wd 


we or. 


specify which of already available instructional materials 
are suitable for teaching any objectives identified in the . 
objectives hierarchy. These materigls: are numbered; in an 
existing! matertals. data base and cross-referenced . to eae 
objectives no eee data bases 7 : 


«a i 
aes 


Design Phase.: Programs. and pace: Bases. Oe nn nn 


For teaching each objective identified in thé OHP, the 
Media Setection Program. CMSP ) provides an.interactive 


‘process for determining a set of .acceptable media 


alternatives. ‘These include any trainer hardware identified. 
in the Procurement: of Trainers Program (PTP), as specified 


in.the Frainer Requirements ‘Program (TRP). These two 


Pes ae to. mainstream ISD activities. 


programs are ‘overviewed in “tfe section on programs 
oo oe 
STs is nea: “passible to, utilize the Syllabus Bava Locman 
Program: SDP). -to* structure the’ syllabus: development ae 
transfer appropriate’ information from the objectives. 


hierarchy data base, and number and reference Syllabus 
elements. The result- is three deta bases: -lesson “level - 


syllabus information, segment-— level > syllabus Le 


cand. ‘a ‘Lesson. puverrelationsnLp matrix. 


* ‘The Training Support Requirements Program (TSRP) acoiehe : 


the instructional. developers in specifying and manipulating 
resource requirements data. The output of the program.is an 


“000 2. a 
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a analysis providing for ‘alternative. “edi mixes “and. “needed - 
. “alterations to basic resource requirement tables, aie are 
ee eeentee for. future use An. the TSRP* Gere base. ; 


Wa ‘ 


IA 


om eo “At. this point, ~the- Relp’ of "the: Lesson ‘Spent fieation os 
_~ » Program-(LSP) ean be' enlisted in. “the: development of ; 
: , individual design. guides which are uSed-‘later to create- sachs 
_,+@etualt lesson. Reference ‘is ’made to necéssary technical, 
Manuals, other related ‘documentation, and tactics” package © 
-. information. ‘Structuring: ‘and tracking of the process is’ 
also peer eres ; . : a ee, " 
fe - 7 : 
ss Thie Implementation Plan beosram (IPP) facilitates 
* “another: important part of the design phasé: This oe: 
-assists’ in coordinating. resources (both personnel and 
+ material) and in’ developing a.complete description. of how. 
the instructional program is to be administered. “All 
, information is retained in an implementation plan data base. 
and is updated. as ether “pt eeram implementation date Bers: lee 
closer. ee re 


er, 


“Finally, the Quality’ Control Program (QCP) dias 
‘developers -in éstablishing @a ‘complete tracking and 
evaluation précedure for all aspects of the developmental | 
process. It helps specify the kinds of data to be collected 
and analyzed. It also helps pinpoint the persons needing 
the data and the time. they need:-it. The QCP creates its own 

. data base as the process is carried out. f . ; 


“Development Bnas cTeeeen ‘and Data Bases 


_ ae _ With the aeepetanne OF the ‘“eecon huthering Program 
..' . (LAP), previously created lesson specifications are, . 
. developed: into complete lessons. The LAP provides help in 
- _ formatting textual components of all lessons and allows for 
. direct. transfer of specifications to ‘the lessons in the 
igsson specifications . and text data base. 


gt Authored lessons. ‘can. -then be produced in ready ~for- 
paste-up copy (or directly ‘transmitted to a phototypesetter) 
with the aid of-the Lessan. Production Program (LPP).. En. 

- « Cases” where audiovisual media are involy ed, this program 

- assists in producing working copies of: scnipts for 

- production, personnel. In addition it can be used to track 
"progress and staffing. All’lessons wh1l be storéd in tRe 
LPP data base’ for easy on-line revision as needed. 


oat . Once .produced in.a sgsaeite form, the Jeseons are given “to! 
typical sample Students- to test Mr. instructional 
effectivengss. | ‘Student. performance, attitudinal data,. and. 
reviewer comments ‘are collected, cataloged, and: analyzed 
through use of: ais Lesson Tryout Eroeres (LTP). The- LTP 
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also assists in the formulation of revision. fbegifications 
“Based ‘en the data-collécted. . All of this information ~ ~ 
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a: 


x 


q 


eacene eae of the lesson validation data base, 
: - ; t 

» rae 
[Inblenéntation Phase. Programs and Data ‘Bases a ae , : se * = Ae, 
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In the -implementati ori phase a Co Site Mananed: . ‘ - * 
pase cuecics Program (GMIP) can be vere to administer student — 
‘tests on-line, to diagnose student deficiencies from test- 
-resulits, and to prescribe student instructional activities. 

This program can also take chargé of maintaining student... 
‘records: (150844. -4 student's progress in—the course. can be 


. - assessed at any time). In’° ‘addition, asséts (personnel, 


‘retained as a data. base. 


materials, Students) are scheduled in -such:a way as to* ; ; 
assure that instruction is administered ins an efficient. and 
ce teceave Manner... pe a Paneer a 

A Computer-— Assisted Instruction Program (CAIP) Gar’ carry 
‘the implementation a step further by actually administering. 
instruction on-line. ‘under computer controlled interactions. 


~~. 


‘Programs Peripheral to Mainstream ISD Activities 


‘Three programs are geared: to deal “with incevnetienal. 
situations requiring complex trainer/simulator wovartenrs er 
-. They are the: 7 | oe | 7a 


aa Historical "Tesiner Data Program (HTDP),; which : _ 
collects and retains pertinent characteristics and’ . 
7 , requirements data pf previously used trainers. and 
, allows for compar face with” current Needs ss - = 
is Trainer Requirements Progra (TRP), which assists in 
establishing requirements. for trainers, along with 
utilization estimates. . : = 


Procurement of Trainers Sroarar (PTP),. which _ Pa i 
provides. for contracting arrangements for new , |e 
_tr@iner equipment or repair or maintenance. of ; 

oe extee ine trainers. - - - ro at, Se 


Each creates its own data base... All three “input: to the. — 
Media Selection Program GMSP) in the design phase. eS 


A’ Tactics Package Program (TPP) provides for ‘the | 
structure and organization of the tactics package. feo < 
doctrine..is referenced’ in the de ana ais hierarchy 
development activity by the OHDB. -The tactics ‘Packege” = 
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N+ 4 Coneurrently with the early analysis phase 
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‘Master Plan Program (MPP) assists in organizing the 


* long-range project schedule. A time line, project ° 


activity, the 


assignmeats list, and \PZRT chart-are created... “All are part | 
of the MPP data base and are-to. be reviewed and updated as. «: 


needed. - 


a Information in the MPP data base is input to they-- 
‘) and. contracting the services of 
‘outside agencies -for particular portions of.the ISD process. 


— : 


A PPP data base is established and maintained.” - 


OA Navy Training Plan Program (NTPP) assists *%in setting .: 
up required NTPP formats and information to-be included. 


The created NTPP data base Will’ be accessible for. on-line 
review and update. 2 we a ee oe 


: ~ 
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A’ Progress Monitoring Program (PMP) is designed ‘to ‘track 


Procurement. Package is eee. This ~program none in 
of,) ° 


- resource expenditures on-a task-by~task basis throughout the... 
“-entire ISD process. Time and capital used on each task are _ 
the» key resources to be monitored and will be~entered into a 


© < . 


project management data base. th 
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DESCRIPTION OF PROGRAM CAPABILITIES og te ee 
_. In this section of the report, a more detailed. °° . 7 
description-.of each. proposed -program will be presented. ad 
Each description will be accompanied by a table listing the. 
types of information to be entered-into each data base, and 


. Specifying the mode of its éntry. ‘A Large portion. of. the 
“snecapabilities available in one Program’ will also be available _ 


' in other programs. herefore, instead, of describing each + 
capability again and ‘again as a@ part of -each program, a 
Single set of these descriptions will be presented. - - 
Definitions of Standard Capabilities ~« 


The gurpose of this section, fs to describe the 
capabilities available in a number of the proposed programs, 
- By describing the capabilities in detail here,.it-will be. 


‘possible to merely reference them in the descriptions of 


ty 


each. of the programs. In some cases, description of-a 

capability presented in this section will be in general | 

terms. ‘The exact definition of the Capability in relation :« 
_to a particular program will-be explained in detail in the 


. @eseription of that, program. 5 
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Interactive Protocols (Menus 
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wherever possible, -on routine incest ves -proteedls which 
cue or,sprompt the user in his or her interaction. For. e3 
example, user interaction with the system should be prompted | ‘ 
by easily under stood cues, such as "Type the conditions oa ca 
ia ead 1.3. 4, 63 wo eg se kn ; i 
. on a ee es - 2 ; 
ae ’ The system. should altow acons De Seiect-ceveeed options. ‘a 
‘from an on-line menu whenever possible, _aS opposed to. : 
expecting the user to type. the desired option from memory. | 
a the _system might ed a @ menu, such as: 
"Do you wish to - a oe wes a ee) 
2 i t; bocg *. : . = 
., write a message to- another user? : 
‘write a message to all users? a a . 3 
"read your messages? eu . ig 


A A “ - ? 7 i 


type the appropriate number . . 2 


This reduces. the typing ‘the user must: do, makes the user 

more aware of System options,: and reduces the user's memory 
load (i.e., users do not need to.bé able to type. well or 
-rmemember complex code words or syntax for Using the wide, 
Variety of system proe Cans available). i 


by “Advice. ‘When the system is designed, ityshould be eae ats 
“mind that the users may be relatively unsophisticated a 
in the .usSe of computers. and sin ‘the process and products . 
involved in the TSD approacHy ” 6 4 we . 


ot 


a 


In order. “to “further ‘reduce che. memory. ‘Load,,the 

training needed by usérs,> the system. ‘should provide. an 

_ on-line "advice" capability. "Advice™ should be available 

“on two- levels. On one level the, system should provide basic 
information onhow to manipulate” the. system. The- 
jy information | given should be as specific as possible to the 

-papticular program. being used. For instance, if=the user is 
using: the review/comment: function, the "advice™ funetion * 48 

~(when requested by the user) should inform thm user how to. 
"asterisk". errors, how to record.comments,. how to -move on ° 

‘when. a" Comment is "completed, how to-.exit from the program, . 

-ete., On. the other‘ level. the system should explain terms, ~ |. 
>.define> ‘input, provide. infornfition as to possible sources of + 

data. ‘needed as basic input,: and give basic’ ee as to 

what | ‘the | Specific program being used does. r example, - if 

a user is, given. the cue "type the conditions for Objective, 

1.324.6," ‘the user should, upon: seeking advice, be given a 
Clear and. specific éxplanation of "conditions" and any... 

related. information as to-standard format to be used’. If 

other information relating to “conditions” is contained in 

some off line document, the “advice” Capability would 


Z reference this source. material. See Si capability. 
Fa oy : Ne 
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Soule “further. assist the. user in te cabins the document. see 
This level of: advice can’ be used in the basic training of. 
-instruectional developers. (perhaps-reducing their training 
- time) and osc some quality. ecnkrot on thé data input. 
“f . 
The: Nadvicet avairabi e should be provided - ‘upon request 
by a simple: command (such‘as a key labeled "advice" or > 
"Hel pt or "info™). In- this way,. experienced. users‘ can a 
. Dypass repeated . reading of lengthy cues, prompts, ete., and 
improve. the ‘efficiency of’their work. Specific. portions- of: 
_ advice should. appear unsolicited when a user makes a 
* | + detectable error. in input, as part. of the. error | 
,reduction/detection. program. - . aS Pk 


tw: 


te 


: "Library. ‘ The Vipeary cabability. Se ehe systen should- 

“S provide a cataleg of pertinent ‘off-line source materials, 

7% and assist. the user in Te cating the. necessary materials or 
personnel. This function will be especially useful in those’ 
_° eases where historical. data comprises a Significant part of 
.' the aa base. 


The library. should astalen metaverse rich relate-to the 
ISD process; copies of previously accep ‘reports, of 
student lessons, etc.; historical data. on. projects similar 
.to. the one undertakeh, ete. It could also catalog personnel - 
who. cauehety as, a cr in some Specific. areas; \e ae a 


a order to ‘faci¥itate ‘finding the appropriate source - 


Poa _ materials the. library. should catalog. all sources in sever.al 


ways--by topic (such as "media selection" or "quality 
control ‘plan"), by title, author, and/or project. When - 
possible, a synopsis. of the material should be available. 
“on-line. Materials should be cross=referenced, ‘and alley 
-entfries should guide users to rélated. ‘topics, ‘documents and — 
sources. . “The > library should also’ include ali. necessary > 
information to "lay your ‘hands on" the desired source 
“material or to. contact the ‘appropriate personnel. 
Error Detection/Reduction Routines. The ystem will, = 
“whenever possible, provide checks on. the data. input. In’ a), 
this -way ‘there will be a-:certain quality control of a}l. data: 
. . Bases.. Most commonly the system. will eheck for- range’ and” 
= domain type errors. | For example, if the only. possible. ‘legal 
~resSponse to a certain question is a number, the user might 
; find that. the only operable keys on the terminal. keyboard" 
-are the number keys. Ali other Keys. would be locked out. 
In this— way the answer. will be at, least within the. 
appropriate domain. '-If.-a tolerance. canbe “specified: for 


“ re 


acceptable. inputs.,. this. too can be checked. . For instance, 

af ait inputs should ‘be two digits, no three-digit responses 

will be accepted. In cases where it is not: feasible to-lock 
& ‘out vance of the Joe ‘or limit ‘the input® field, the | 


\ 
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_ only be input one time. If_that same piece of data is-a 


: 


system’ will ‘prévideferror’ messages * adver: cues to srompe 
immediate correction of-errors,. whenever ‘an analysis: of user’: 


re a ae # NAVTRAEGUIPEEN ATE 0018-1 ~ fe oes 
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-npue: can detect and classify an.error., 


+ >. 
bad ° 
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December 1, 1977 and is’ input as Dec, +, .1977,the -syStem. 
‘should note this error and either ‘automatically: eofivert 


“Dec.” to "December! or alert the: user to make-the. necessary: 


~change. The system should automatically check ‘that no 


‘ mMon-month word was used and that no illegal day sa coe 
Gi. e., December 4 Bg 1977) was input. | Fa 


~- 


Automatic ‘In ut of Data From previous Inputs. The various > 
programs within the system will be cross-referenced and 
linked in- such'a way that any specific piece of data need 
necessary and predictable part of the. data-base of- another 
program, the system will automatically retrieve that data 
piece and use it as’part of the new data ees a ies 
instance, “the objective of any given piece,of instruction . 
will be an important piece.of data in several | ‘programs.. 
Once the objective has. been input into one=program, the 
system will automatically retrieve the necessary objectives, 
use them aS part. of the new data base, and aesphay. them as 
needed in: any. and. all reports. 


: Where such ‘an automatic and predictable ehossereterence 
-is not possible, the-system should offer a range of support. 


' funetions (copy, edit, move,-etce.) to help. eliminate 


redundancy of manual effort once oe Cote. exists 


Somewhere (anywhere). in the system. 


deenatuned “Input .. ne say steer will define ae data to be 


input ,. -assist the ‘dsers “in collecting and organizing the 


. data, and will. structure the data bases into easily used . 
tables, charts, “tists, and eponts. 9) a . 


LS Raley 
The user wai: be assisted i collecting Zand Srganizing: 
‘the Satelit data in one of two ways. . eS 


ae Wreré the data to be collected is: consistent across. 


nearly. all: content, form-driven programs can be written. 


‘Printed: paper ‘forms can be provided for ease in co lecting’ 
the necesSary ‘data. Clearly spelled out pr cedures, ill- be 
given as: to. ‘how to correctly fill out the ‘paper form when 
collecting data in the field. When the aper form is 
correctly and completely filled. out, ‘all. 
“be available for input. Input, into e “system will. require. 
merely replicating | the information oni.the paper forms. In 
ata aes SOnmS can be machine. readable. The paper forms 


° sat : > : oe ae : : eo 
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In. ‘hose data bases where speci fic syntax and eoreae' are 
‘critical. elements, the system should make reasonable. chécks 
.on these factors.. For instance if. a date.must be written as 


~ 


ecessary data. will. 
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oe . . NAVTRAEQUT PCEN 77=C-0078-1, . wee 
“wc J “should be easy Lo .use ,and ality gnstructions should 6é. clearly 
"| Specified Ceither on’ the’ form-or bn an accompanying: job aid), 
a - @S to information to be eollectéd and the vcorrect syntax to * 
' be used. Syntax and format of.-the: screen display and the ,. 
: definition of acceptable input into the ‘system should bé as . 
«he . Similar as possible to the format and syntax used. by the. | 
' Paper forms. ‘eg eee eae a ee 8 | 
a, - ge ee eS et ee es 
"2. Where the type. of data ‘to, be collected varies . = _ 
considerably across different content, sinteractive, se 
“branching Programs will-be -written. ° fn these cases, ‘input 
‘2 Of 2 Specific piece of-data will cue the system ‘to request. . 
bd ee of the appropriate next piece of data. For instance, 


Fs 
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ifa user specifies that an objective has been. developed iNew. 
. .workbook format, the system might request data Such as the ‘*\ 
' number of Pages. Om the other hand, for another objective 
+ taught by -slide/tape, the ‘system might request-length Of 4. 4 
. presentation time.in minutes see A al of slides produced... | 
- Onee again, clear-cut guidelines as to t < be 
a used in collecting the. necessary data and/or criteria for 
#4, 1 Making. decisions will be prowided.. a 2 
_! og.) Imput ‘from either fonm-driven-ddta bases or initeractive, 
oe branching programs will be structured by the system into. 
various usable ata bases which will be available to the’ 
user in the form. of tables, charts, lists, and: reports 
viewable on-line or in hardcopy printouts. 2 = 
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| Automatic Numbering and Referencing. The system wills -when 
_™. appropriate, define the numbering and referencing system to. 
be used. For example, objectives hierarchies would use the: :. 
Standard military system where Objective 1.1 is subordinate | 
to Objective 1, and Objective 1.1.1 is then subordinate to? . 
4 1.1, ete. AlI syllabi would use the standard unit number, .: 
lesson number, segment number. By defining the numbering — 
and referencing systems to be used and requesting ‘certain-. 
data in a Predefined sequence, many inconsistenciés and , 
inaccuracies in references can de avoided. For’ example,. the 
System would.prohibit input of two ab jectives with the same | 
reference number... It could be Programmed. to check and . . 
prompt users that Segments;listed in objectives hierarchies 
aS prerequisite to other i Struction should be taught in an 
' appropriate Sequence, Ci.e.,. not in an order contrary-to . 
that indicated by the’ objectives hierarchy)... 6° 3. 5°. -, 


caused¥by deletions and insertions can be very burdensome. =<. | 
- Such changes should be provided automatically dy the. system. — he 
’ For example, if the syllabus for a given lesson indicates in 

a given lesson that there are four.segments and it -is 

decided to insert a new segment,. (.*) between Segments 1 and 


In large data bases, the renumbering of the entries 


ogre wey : 
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“3, “C1, aes un, a leaving. the. néneinine sequencé the . & 2 


any legal entry in the third field, and 7endane. ‘with. Oe 
‘ - ad 
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“same, the - Ei qn’ ‘Segments. showld be automatically = renumbered 

seRy Mold¥ segment 2 -b mes 3y 3 becomes , etc.) oa 

Ca ra 

The.gystém shouxe alsoeprovide appropriate Linkage and 

cross-referencing. of data. For instance, a. given objective -. 

may appear as 4, ‘prerequisite: or enabling - objective in more - a 
han one hierarchy and, theréfore,- have more than one a. i ‘sf * 

hierarchy number. The same object-ive would be given ° 


'..different numbersS'in the final syllabus. The system must.. 


cross-referefhce and “link identical data found in. different. - og 
data bases. In addition, the specification of identifiable 


: subsets of data elements through-general. identifiers (i.e.,. va ee 
'1.1.%.6) should. be possible. This example given might mean © ° 


to select every four field entry beginning with 7.1., with - 


as 


Hierarchy of. Users. : The system. will allow” a. hierarchy ef 
users to ke specified [i.e s) ‘certain “user's “Can ‘ev prohibited ” 


.. from, using.certain system capabilities; specified users can 


“og: the apERopesers: procedures. for making changes. . 


be limited to the: use of certain capabilities or.” "bo certain 
programs; and input can’ bé tagged as to the, user Cor | 
authority of user) making the input]. For instanée, the _ - 


' reviéw/comment capability might be restricted only to,.those 


authorized to review the report. being dealt with. ‘The —. 
on-line update capability would be further restricted té the 
use @f those authorized to approve changes and knowledgeable 


ro 


On-line Review/Comment. “This. capability: will allow 


. Authorized’ users through the terminal keyboard could then 


“authorized users.to review on-line (i.e., at a termigal) the 


data input or reports generated by the related program. 


Tasterisk" or indicate in some standardized way areas-:where — 


‘they deem that changes are necessary or: problems- arise. A, 7 
mechanism will be provided whereby the user can then. “#. * > ecg 


"comment". on each "asterisk" (i.e., explain specifically ‘phe Fe 


“signature. The signature could alsa be left to the-user's 
‘discretion. Comments could be ranked according to the user 


proaee identified, propose solutions, ete) . 
The "eomments" could: be left anonymous ‘or require a 
making themiCi.e., the comments of the person charged with — 


the responsibility for réview of that program could be ~ 
tagged in. some way -so as to quickly. distinguish these 


comments from. these of other users). ; - . 


“Coumenve could also. “be categorized according ta vihe type 


oe error or problem identified. “For. example, after a 


inputting an asterisk, the user. ‘might’ be asked to classify 
ene error as’ oe . . eon oe _ 


ERIC - < - * (ee : Pea ~ 


ey 


x 


* * o 
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typographical or spelling error. 
*. wording change: 7 8 os 
e. content change. : _— 2 3 _ a e 
erners : a a os = ; : 


i? ta: “thi's way, spelling errors “eould he. _quiekly adeneieieds 
and. changed without further .ado, while suggested content. .*. 
_ changes. co id.be reserved. until any necessary approval. for. 
such changes had been received.” A mechanism would be 2 3 
provided whereby thdse who were so -authorized could review. 
. and comment, and indicate approval or. | aed ttae of ; 
- Suggested changes made by others. 

The: reviGw/comment capability would not allow users to 
“permanently change any. data ‘input ,or any information , 
ineludéd-in'a program report. At this time all comments. 
would be. considered suggestions, although some comments, 

aad could be routinely accepted for change either because the 
error was routine (a spelling error requiring no approval), 
he user making the comment ‘had the authority to authorize. 
the change, or the comment had been approved. by another | user 
with authority to authorize. changes. a Z 


On-Line. Update. THiS: capability woul’: ailew: authorized’ 
> users to make permanent changes in data previously input to 


, the system. All text’ editing/word processing capabilities 
' would be available in making updates and corrections. Where 
a Single piece of data-(sue S an objective) appears as. 
part of several data bases, a change in the. input. to one . 
data base shculd automatically. change. that piece of data in’ 
‘all other data bases4 unless those data bases contain a 
_' modifications of the basic data. In this case-the_ j 
. modifications are called to the user’ Ss. attention: for” 7 
. decisions and. actions. - For. instance, if the wording of an. 
- objective: is to be slightly changed in.one data base, the — 
> system. should. automatically. ‘and consistently. make the - ‘same _ 
-echange. in other ‘data bases. in which the: objective appears... 
All objectives needing a similar. change should be readily ~ * 
located “and reviewed by key word Searches. Where a change . 
ina Single. piece of data: implies a consistent and- epseisie:, 
change: throughout, a data base, the system should . 
automatically make the change. - _ 


the ‘system. could alee keep certain records useful. in 
quality. control and. accountability procedures. -For:. | “ 
.instanceé,. Tt. could\keep ‘track of the dates on which a data: 
“change was made, who :made it, recommended it, authorized it, 
ete. | Ee Por 


% 


data. bases and Prone. multiple. coptes therdo 988 nrerary 


" ane . 


a. *S _ s 7 . ; 2 / 2 e “ ; q 9 H a 4 ? 3 . 
Bee eng! 3 : — a fe a 7 
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Bois. funetion allows the user to ‘play—twhat.if" games. 


‘These "management models" ” are extremely important in~ making. 
accurate projections or complex, decisions on time ;, budget, 
personnel, -and other Management. and instructional concerns... 
,For example., ‘the actual course being developed” might have™40 _ 
*videotape objectives specified. If it‘is found that the © 
-project is Funning overbudget, one alternative to change 


‘might be some of the videotapes to the second ranked media 


. 


choiége, slide/tapes. Using’ a temporary file, a "what if" 
data base could’ be established showing all videotapes as. 

Slide/tapes. The computer could then project. {using data 
input regarding personnel needs, ‘cost, time ‘requirements, 
ete., for developing slide/tapes) what financial savings . 
‘could be affected by changing these objectives to : 


.Slide/tapes, what personnel changes would be needed, when 
‘production could be completed, etc.* The on-line "what pad 
‘¢apability allows faster and more objective evaluation of 
_ alternative decisions than ‘a manual oe cea ‘ 


we . 


—— 
. Communications Facility. The system should have functions — 


permitting the users to. communicate with. each other. 


Functions ‘should be designed -in. such: a’ way that one user can. 
communicate ‘with any other specified user, a subset of users. 
(such. as all Instructional Developers), or’with all users... 


~ The system should allow storage.of messages in such a way as~ 


to prompt uSers not. currently on-line to check their. 

messages whenever they log-on and reply or respond in 
whatever way is appropriate.. This- function should help. in. 
efficient production and save considerable time. ._ For’ | est 
‘instance, an authorized user who has completed a review of 


_certain material. could notify the appropriate personnel of’ 


the. material reviewed and in this ‘way the material-can- enter: 


ee next phase of production, as’ quickly as possible: 


files: eee See then be manipulated and:. ‘changed at wilt dot 


Structurea Report Format. When the format, Headings: ‘and/or: 


wording to be used in @ formal’ report can’ be. specified, the: 
system will display. the format to be used on the .screen*‘and 


leave’ blanks for specific pieces,of data to- be anput. ' The 


user: will « only be required £o epyeerd the. specific missing 
data. ——— 3 


- 


eo ‘This function could be written’ for. two ievel's of users: 


ce Unsophisticated or inexperienced users.” Oe es ee ea: 


re 
io this version of the structured. report format, the 
‘system ‘could number each. “blank” ‘left in the Feport and “e 
users.could. indicate by~ ‘number which blanks they were ready | 
to fit in. . The ‘user: would then ‘receive a number of cues as- 


tho 


] 
i 
t 


e yoo. 
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+ 
ef 


t 


to the specific“inrormatYon: belonging in the-blank and as to 


might lead the user ‘to a-cue such 


"On what. day is the report to be 


the correct format.to use. “For, instance,-blanik’ Number 1 - 


BS ss om <% 


‘.).. submitted? Supply. the date in this 


bel, ge Pee form: December V;. 1977" 


Peer 1 


os It is implied. therefore, 


automatically check for this format, no noOn=month word would 


>>, 


that the program, will 


be accepted, and no illegal day number (i.e€., December 45, 
1977) would be allowed. Where whole sections of the report 


/are to be supplied,. the instructio 
more general,-but would provide as 
possible. ‘For instance, the instr 
be: ; 7 an , 


"This section should expla 
man is the rationale for the proj 
a proposed. Usually the see 

. five paragraphs long and d 
Personnel needs of the tra 


ns would necessarily be 
Much guidance’ as. 

uctions to the user might 
LF , 
in in°general | 
ect plan being 
tion is four -to 
eals with- the 
ining program, 


the availability of the personnel needed,. -~-. 
Proposed cost, and projected ie 


improvements." | % 


' & When a user has specified 
the system would insert the new da 


bases where it might be ‘needed). 


- 


any needed information, - 


~ 


to 


ta into the appraepriate- - 
position in the report (and. incidentally, .in any other data > 


2.°. Sophisticated or experienced users. - —— 


This version.of the structured. report format would 
allow. the user to bypass cues, about needed content and . 
format ‘ahd to supply only thé needed pieces of information. | 

- The user could type: the néeded, information into the report : 
@ . itself. The user would need -to. be provided with’ simple : 
al Space, for’ moving back.” 


mechanisms for requesting addition 
nd forth through the report, and 
aktivities.. _, o* CRE 
Any user should be able to 
uSage. level to another; 4s he or s 
writing one part of .a report and i 
another. In either version, users 
any point how the data input appea 
report fdrmat and revise as necess 
Reading Grade Level Computation. 
Or actual instructional materials: 
function should be provided to ass 


for. other editing - | . 


ul ‘%: 


move eaSily from‘one. 

he might be experienced “a 
nexperienced at writing 
Should be able to see at 
rS in the -structured 


oye | (ge . oF 
Where sections of reports 
are to be Kept on-line, a 


ess the reading level of 


oe ee. Ht, Oe 


- i) g : 
ERIC ' : a ; 


\ 
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any desighated text. passage. ‘Severa 21 goad algorithms exist oy 
for this-funection, and one | sneule: be adapted rather than’ a 
reinverited. = 3 an — ; a 4 


2 < ‘ a - 


This function should be able to analyze any identifiable - 
- text field on the’ system and return an estimated reading 
level. in real time.. -"Advice”™ or "help" for ‘this function, 

in addition to specifying how to use it, would briefly. 
describe how it. works and-would offer a few helpful. ‘hints. in 
reducing the reading difficulty of a text passage. 


Text Editin /Cu -File Maintencnce ‘Su ort. This- 
capability would Sliow users to.easily and efficiently _—. 
correct and change: data presently or previously input into ~ 
the system. It. should be designed for simplicity of use and — 
should require minimal training. for ‘effective use. 


Pal 


Some of the capabilities of this. funetion include: 


1 Request for margins to. be automatically right and/or 
left justified. 2 : 


2. Ability to scroll forward -or besxward through a 
structured report. : ; 


= @ 


Ph ie Automatic. page number ing of eepistured reports: 


ae ebonat ie lire count for. page breaking of structured 
.' reports. ; . 


5. - Simple | éommands for inserting additional text, 
‘.deleting: text, eliminating , unnecessary. blanks 
, (close-up), rearranging blocks of previously typed 
text, -and automatic .reformatting of subsequent text. 


ome ee rere storage, and easy ‘peer eeual roe often 
. used words, phrases,: and paragraphs to prevént: the. 
: need for typing: the same text repeatedly in’ To 
_ different coutexs (copy/duplicate). 


Te ‘Specification of and automatic changes in tabs, 
indentation, and- melee adjustments. 


oy. Automatic wenterine (both horizontal aid vertics we 
and left and right justification. 28 la 


Be Atitomatic decimal alignment. 


-10.-Automatic-search and/or replace for strings or 
: automatic-stopping at formatted input points. -< 


e i ae .. 
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. ees commands for indicating color déaee changes in 
font-Cseript, ‘boldface,~superscript, Subscript), or 
other text attributes... Sg 8 


¥ a 


12. Automatic underlining, uppercase. 


Herdecosy Printout. - This capability will. allow- the user to 
‘request a-formatted paper copy-of a specifics data base’ or 
report generated. by the CATSDM: system. Such paper copies 
are often useful for review purposes and provide valuable: 
historical records. | : | 


_ ot S 


HaPaeepy printouts could ‘be. of. three. basic ‘types: 


1. A hardeopy printout of a ‘single sereen “display - 


Any ‘display or part of a display: appearing on the. 
computer display screen could be ‘printed. . The use. would not 
, need to spend time taking copious sme tes" on _ sEeOrm aero? < 
- displayed. - 4 ar oe 


a ; ar 


2 A listing from a Yata base. . / Se 2 


- Once all related data bases had been anput, the, user . 
es could get a Structured, easily understodd printout of a 
specific data base. For instance, once all pertinent al 
information.related to media seléction -hag been input, 
-user could request a listing of all objectives uSing ~ —- 
videotape. A series of menus could allow the user to 
a ; epecity the Specific Subset: ins pmie , 


- 


— 3. A printed, structured: report format/report: = 


a eee ‘Wen/s report format. ‘can. be speci fied,’ the: ‘system 
7 ean: “provide a printed version-of the format, supplying all 
oN standardized ‘wording, divisions, etc., leaving blank areas ~ 
for any. specific information to be supplied. Once’ any: a 
a “pertinent data to be ineluded in the report is supplied’ to -_ 
-the system, a printed version of the’ a nesud ene Paes ; 
“new data, can be requested. : . 


e 


oo 


Progress Tracking. ane progre’s | tracking function of the. 
? system is an on-line management. tool. It allows users. to 
break majon tasks (such as‘developing a final Navy Training 
- Plan) into subtasks, to assign each ‘subtask to different . 
-- -. personnel, establish due dates, and track. progress (noting 
such things:as who has reviewed the data input, who has - 
, revised it,-when these tasks were performed, ete.). This. -. 
“Capability will facilitaté lower-level management functions 
‘where it-is less likely that Enapeeeotad or. eeperiencee 


: ‘managers will be available. - > 

; . Soe - {= = 

= ss a 25 18 
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a ee dard bate Base Mana erent Suneesoes. The CATSDM* gece. 

oo d needs to have a data baSe management ‘system which will. 
Clude such attributes as keyword searches, indexed access, 
mone a adaacG of. data, data security. at various’ levels, data» 

base creation, data base. updating,-inquiry, and report 4 
generation. tt will provide . for field Validation on entry, 
update and error recovery, and sorting of the data base. 

: Standard CODASYL specifications and standards for Data Base 

eee Should be followed to the maximum extent. : oe 
possible, in order to increase ene level of standardization 

Petween systems. ; ; 4 . 


” 


Description of Functions oe Each Program: | ng a —- 


This section of the report will describe in some detail 
the purpose and’ furetion of each program in the CATSDM 
- system... Each description will be supported by a table. 
listing -the exact types of information to be collected as 
-part of the program, how it will be manipulated, and in what 
data base it will be stored: . For organizational. purposes | 
these tables. are linked in this document directly to the 
appropriate data hase description, and so appear in the next 
section with the oes bases. 3 fF 


Analysis Phase. ‘programs 


‘Problem Analysis Program (PAP). The Problem Analysis ~ 
Program will sassist the user in the collection and. -- 
structuring. of the information necessary to determine the 
program: goals ,. constraints, resources, and problems: which | 
will shape. the. ISD process. The program will prompt” the 
user to gather. all the. eategories of ‘information. needed. 
. (such as mission characteristics, weapons system. design 
> @haracteristics, existing training facilities and equipment, 
‘Pp existing schedules, funding .levels, and réquired end product © 
characteristics). The system. will also pve advice as to- 
penstnte onreeS of such data. 
The program will allow on-line storage Ee the. data and 
_ will organize and structure it so that it can be most cla daar 
_ retrieved ane analyzed ‘by the Users. 
_ The: program etd need aohstant patie aha should _ 
4 -therefore have ere reveew/ comment. and edit/update 
capabilities. “a ee ’ 


je fors 


The. final’ dutput of. the program is a Problem Analysis: 
Report. The system Should support tHis effort by its: 
; structured report format and text editing/word processing 
~ Capabilities. Previously developed Problem Analysis Reports. 
should be available either on— or off-line as references and... 
be: easily bo¢ated using Advice and | Library system. | _ a 
capabilities, ; * 


LS Sn rr a re ar ne ee a TO ies 


an 


_ Student Entry-Level Program (SELP) 


Structured input and data reporting. 
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Task Listing Program (TLP). The Task Listing Program will 


_ assist the instructional developer in the-analysis of a 
- Given job and in the, development of an orderly list of the 


major tasks to be performed 5y an individual working in- that 
job. The TLP will help organize the analysis procedure by 


_On=line cueing. and prompting and by ordering the input of 


the data base. .The user would first be requested to specify 


-all responsibility aréas (or systems). in the job. ° The TLP 


would then have the user select one responsibility. area and- 
mentally walk. through it, listing each mission (or 


. Subsystem) related to that area The program would guide 
the user through the -entire analysis procedure ‘until all 


tasks performed under both’ normal and extraordinary ; 
conditions were identified. If at any point the user were . 


"unsure of the input expected, the TLP should provide .an 


explanation of the task listing development process and ~—- 


_ definitions of.terms, etc., upon request. 


2 2 


_ -The program would automatically provide the standard 
numbering and referencing system: familiar to the military * 
community where Mission 1.1 zs subordinate to “Reponsibility 


‘Area 1; where Phase 1.1.1_is part of Mission 1.1, and .so 
forth.’ - a ee Te gg ge ora 


After all tasks are identified, the TLP will request: the 
user’ to specify conditions and standards for each task in a 


Standardized format. . Oe “, 7 a ee 


‘The TLP will have full use of on-line review/comment and 
updaté capabilities. These will be especially useful during 
the validation phase of the task listing development vs 
process. The TLP will, therefore, need a full range of 
editing capabilities. Orie of the most: important of these 
capabilities would be the system's automatic restructuring 


“and renumbering of the tasks listed as tasks are deleted or 


added to the original listing. The program will require an 
automatic cross-referencing system based on keyword searches 
to permit easy identification of identical or related tasks 
falling under different responsibility areas.. - 


“Much of the data base iniput as part of this ‘program. will? 
need to be tagged and automatically supplied as part of «. 
subsequent data bases for such programs as the Objectives . 
Hierarchy Program (OHP). . wt eee ee 
The Studént Entry-Level 
velopers in determining 


Program will..assist instructional 


“and describing the characteristics of students coming into 
“the course(s). It will indicate what data types need be _ 


obtained and how to find. them.. It will also provide for 


ae 
‘ 


; ae oe ; 225 p> as 


a 
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Tavereornue protocols will be” employed to ‘help ene user 
and to determine what student_ entry characteristics should 
be described. These may inctude previous related training a 
“Cineluding terminal.objectives mastered), aptitude 
‘indicators (such as GCT. scgres), -related job experience that. 
is required and/or can be Substituted for training, . and 
"other useful data (such as age or pay grade). Much of this 

~ information will be available in the Problem Analysis _. 

“Program (PAP) data base, which will be accessed” through this: 

| program,. In addition, :the user,will be given suggestions as 
to where to obtain the: necessary data, and a checklist for 
tracking his or her efforts.” ee SK gE Lage 


Structured agus capability will ensure that any entry | 
level.data is entered correctly and numbered automatically. 
Terminal objectives’ from previous training will be numbered 
so that they ‘can. be cross-referenced as prerequisites to 
‘objectives in the course under develapment at the 
appropriate point in the development process. Entry level 
data will also be entered so developers can see.the range of 
_@eharacteristics of the target -atididence, as well as what the is 
typical or average student lookszjlike. . For example, it + 
might be found tkat the average Student has a score of 76 on 
_ a given aptituasfeest for which the student population —- 
scores range, frém59 to 92. 2 Other.test parameters should be 
available as_ “needed cee g-, mean, Standard deviation, etc. a 
:  Steuceured report formats: will be available for Creating 
hard copy printouts of entry level data (student entry level aa 
report) in a form that can be utilized in the development. id. . 
effort. The user. will also have access to previously 5 ae Oe 
"generated reports. -to review as. examples. © . a 
Task Validation and. Selection Program (TVSP). The Task 
' Validation and Selection Program will.assist the 
instructional developers in determining the validity of the 
tasks included in a task listing and in sorting those tasks 
‘into the following training categories: ; nos 


. 1. No formal training. «.. . 
ise Review training only. - i ee ee |, 
= Familliarization. training only.  & e ae = See 
Fie ge 4. “Formal training at FRS: * - eee ge a te 
2 pe get Training at operational “squadron. oe Rite om  & 
“F* gnee the instructional development team nas determined 
‘the task selection algorithm. and the exact questions to-be . 
asked during. the task validation” ‘Survey, the TVSP. will oo 
assist in formatting and printing out. the necessary survey ae 
“forms by using the program' ‘'s text manipulation. and hard copy 
'“\»printout capabilities. The instructional developmént’ team 
will determine the appropriate groups. to nS surveyed: a 


—— 


‘the program will either €a). administer the survey on=line at 


ee me - 
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all appropriate sites (prompting. to. allow maximum-ease of. 
use for what-is most likely a group of inexperienced users), 
or (b) allow for. batch input of survey data from ee eee o 
appropriately designed and compeleted survey forms when the 


Survey must -be: conducted in the field through.-use of 
~ Paper/pencil’ forms with no access. to computer .terminals.. 


. The TVSP will analyze and summarize the survey data, 
Structuring the information into a collapsed, easily . 


understood form, including a catalog and summary of comments 


made by those surveyed. | . oe = 


. 


cin*analyzing the..t@sks selected. for inclusion in the 


. megessary the numbering and referencing of subtasks, 
~ subtasks* produced...) 


-instruétiénal: 


— - er Ge 4 og : 
_Onee the validation. survey data ha's been input,” 


training and sort them into the categories. described above. 
In. order to make such -selections, the TVSP.will use the task 


_ selection algorithm developed by the instructional 


development team, the survey data, and any necessary data 
from the Task Listing Program (TLP) and the Student Entry 


. Level Program (SELP). 


The selection of tasks done automatically by the TVSP 
will be a preliminary or "rough cut" of the final selection 


of tasks. The instructional ph gece bel igh tots make  °. 


refinements on line by using On-line review/comment . 


using the editing and update capabilities. 


Objectives Hierarchy Program’ (OHP).. The Objectives. a 
Hierarchy .Program will assist the instructional. developers 


, 


training program and in developing the required cbjectives 


and. hierarchy diagrams. ae . 
= ae ; . 


The ‘OHP Will provide on-line protocols to guide the. 
instruct al developers in listing, in a logical. and 
orderly fashion (with advice upon request), all component 


the various:editing “functions, the user should be able to. 


_ add, delete,-or consolidate.:subtasks with relative ease, 


The program will automatically-provide and. revise 4s. 
use of the standard “military referencing system. Authorized 


ew the .T¥SP | 
will assist in‘the> preliminary selection of tasks’ for. 


wk 


. 


' subtasks, including decision component ‘and récall component ~ 7” 
-subtasks:, for each major task ‘selected for training. Using. 2.5. 


' function. Authorized users will:make permanent changes by “ 


Ea 


on 


through ~° 


users should beable to. review/comment ‘on, the list, of... ee 


~ 


‘The OHP Will-also provide a means for input-of _ a 
Objectives for each task (behavior to be. j,«= 


_ performed, conditions of performance and minimum acceptable — 
_performance standard).  . 2. oe mo eae 
eS ee : ih ae BH 2: en 


wt. = * : 
2 ec ; : » ae We . 2 d pire 
‘ 


a. Gene sora ; 2 7 : - < 
a ta, o . ie oe E aa sat LN 
: wel © a 2 7 eee & : 
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The. program will automatically ‘tag Seen objective inputs 
with 2. reference number by using the standard military 
referencing system This system indicates She subordinate, . 
“ superordinate, and. independent relationships existing , 

: the various component subtasks (Task 221.1 is» . 
: sto Task 2.1, ete). Tasks and subtasks- appearing 
‘more. than ‘once in the hierarchy will show alternate numbers. 
to ‘point: Ot the existing a acai auc its. place: of 
occurance.” 


Fut 


Given the Aaa! “stated: above, the program will produce 

. the necessary ebjectives hierarchy diagrams for the course 
being developed. It will also provide a catalog of 
inflermation related to each objective to be trained, 

includ ing ° necessary cross-references to the ee ee Task 
‘Listing,: -the Task Listing Validation, and the Task Selection 
, Report. The program should provide an automatic search ‘and 
-eross~referencing System to enable “the instructional oe 
’ deve¥opers to easily find.identical subtasks appearing in 
different parts of. the objectives hierarchy... The. program 
should automatically. tag such objectives on. the output 
diagrams: as well’ as in the. catalog. : — 


. To accommodate: the. necessary. changes and updates to. “thes, 
objectives hierarchy. diagrams, a full range gf: 
review/comment, edit/revise, and manual override of program 
en and outputs cee Stes . 


‘Existing Materials Evevuat ion prope smn EMEP.) “The Existing. 
Materials Evaluation Program assists instructional- ae 7 
developers in.collecting,, evaluating, and cataloging . 
existing: instructional. Materials for. possible, use in the. 
course under development.. iThe result of this activity is a 
list of ‘appropriate materials eross-referenced to objectives. : 
in the objectives hierarchy. : 


! a a é 


a a 8 “ 
‘It is first necesSary to determine. seseibie sources of 
eyesei pe related materials. | -The EMEP will display ©. 
interactive protacols enabling. the user to determine generic - 
content areas addressed. in the’ objectives” hi rarchy.. It.. = 
“will then. advise the. user concerning p licat ions, 
‘directives, and: ‘established. training. fase 
| training in those generic. areas... This m 
locate and request materials for review. 


a= % - 


Kes ws 


Once materials are ‘obtained: their lea iA oh seetives + 
can be compared to those in the objectives hlerarchy data 
base for’ applicability. . Suitable objectives - ‘can be entered. 
into. the existing materials (EM) data base and automatically . 
mumbered and cross-referenced to. the objectives hierarchy... 
information 2 rom the: ‘student entry tevel Gate neec can. iso 


ies fete, SS Se oe 


-* wavrra 


_ be accessed and’ compare 
| existing materials to qd 
.,8radeé level are appropr 


: The EMEP. wikt provi 
quality of existing mat 
—Bpelyzing available stu 
or jectives to instru 
“. Criteria, and: appropria 


materials will be retai 
the on-line update capa 
“reviewed and added as n 


Design Phase Programs 


| Media Selection Program 
will assist the instruc 
consistent and instruct 
decisions on-what type 
given objective. 


«=. -The instructional d 
ee appropriate reports. and 


". . media and Will thén' dewelop an ‘algorithm to sort*the’ = | °/ . 
- “objectives” into the various media categories... The. Library: 2’ 


Capability:.will .be usef 
“interactive branching p 
media selection algorit 
necessary input on whet 
(study session) type .of 
session) type of object 
'. priority. decision rules 
'. optimum delivery medium 
for each objective-in t 


6 
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d with those characteristics of the 
etermine if such things as reading 
late for the target population. - , 


de guidelines for. evaluating the 
erials. It will assist the user in 
dent performance. data, corr@spindence. 
Ctional strategies and evaluation 


“numbered and cross-referenced ' . 
ned as the EM data base. By using’ 
bility additional materials can be 
ecessary.:- : So 


~ 


(MSP). The Media Selection Program 
tional development team in making 
ionally and economically defensible © 
of media to use for teaching any 


oc 


aN 


evelopment team will examine the .. 
documents to identify-available._ 


ul in this effort. ‘On-life9 = |. 
rotocols“will guide users through*th 
hm-and provide the prograg/aith a 
her, each objective is a classréom in, 
objective or a hands-on: (device Cs. 
ive, etc. Once the necessary .- 


are input, the MSP will-identify the - 
and all alternative acceptable media 


he objectives hierarchies. |" 


Joue' The MSP “should” also..provide. information regarding the 


number. of objectives se 
lists of objectives and 
media. 6.3% 3 
When available inst 
- defined, the-MSP should 
‘requirements of "to~be- 
summarizing sékected-re 
algorithm into a design 


agi fot 


= 7, 
ERC. 3 


lected for. each medium and. provide | 
-related information categorized by 
a: P nae oe : . : 


-  f 


ructional-media are not clearly. 
hhelp.specify the instructional © 
acquired" media by cataloging and 
Sponses to the media’ selection 


report. For,instance, if trainers 
P 29%) . 


* ry P : 
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have “oe yet - been. designed for a.new weapons Sy res. the MSP 
program should assist the trainer. designers to determine 
what capabilities the trainers must have to best meet the . 
training needs of future ‘students. If a ‘decision must be 
‘made as. to whether workbooks need: color or can be: produced. 
“in? less expensive. black and white; the MSP should be able. to 
point: out. those workbook Sojectives where color+$is- 
instructionally critical and which alternatives are 
available, etc. Thus the. decision. made will be poem 
instructionally and. , economically. defensible. -. 5 


Syllapus- ‘Devel opaenes Program. (SDP). The Syllabus’ 


Development-Program should assist the instructional  ~ 
, development team in developing lesson “maps, ordinal. cour Se- 
‘Syllabi, and. time-based syllabi. (class. schedules). 


* 


- 2 
The SDP should provide on-line, inter sgeive protocols 
which will structure.the material and guide the’ -) 


instructional developers. through. the process of sequencing 
and organizing the objectives. Selected for training into a> 
workable’ course outline. The program Should assure that the 
- final organization of-the course allows students to (a) © 
proceed from easier to more difficult Material within a 

task; (b) have early hands-on experience. whenever .possible, 
and Ce) address.the most critical. job tasks as early as 
-possible.. In. addition, objectives snould be grouped into 
lessons -in such a way that (a) use of trainer-requiring eS 
lessons is.made as efficient as possible, ’and.(b). only one,... 
or possibly two, presentation media. will. be needed within a- 
Single .lesson. Data from the Objectives Hierarchy and Media 
“Selection programs will ‘be. essential “in- ‘determining ‘that. all, 
Wecessary Factors are considered (data an De on} 
‘appropriate media, etce., from the OHP and MSP should be. 
readily available to the user if requested) so decisions 

made during the SDP are based. on aecur ate data. The: program | 
should provide some sophisticated quality control by 
cross-referencing with the OHP and MSP. For example, the 
program should. automatically check and cue users if a lesson 
Map appearing inthe course cutline sequences objectives so _ 
Ss: to violate the subordinate/superordinaze relationship 
specified. in- the. Objectives: ‘Hierarchy Cive., so students do. 
“not receive Necessary prerequisite objectives first) - or .if a 
“Lesson: ‘presentation medium. is. selected whieh is . 
pigperopr sats: for’ Cer eaan: objectives. aig ed in the Lesson 


The ‘program should make extensive use of the automatic 
input function which ‘Will automatically copy significant 
“pieces of information from the OHP and MSP data bases re. 
the data base needed for the syllabus. For instance, the. 
user should not need to-retype the conditions, behaviors, or 
Standards. for .any Objective, as’ these were previously input.. 
The program will also provide an. automatic numbering Cand ~ 
ee eames ane reference system. 
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= When ail necessary. dopa is input, the eect should: © 

. produce, in a specified rmat sson and unit \maps - 

’ .Cdiagrams) ‘indicating Suge srd ace subordinate ‘and 
independent relationships between objectives. It) should | 
also provide an appropriate ordinal sequence for progressing 

°- through the training program, as well as lists and/or ' 

catalogs of all related objectives and related numbers and 

- references, suggested lesson media, time estimates for. 

Student i ah and other pertinent -eross-referencing — 
cae 


Once. the-ordinal Syllabus has been Geet aves the 
program will assist the user in developing a suggested daily: 
class schedule for all training programs and overlapping 
sessions of each course. This will be done by interactive 

ranching -protocels and automatic retrieval of data from ., 
“ether data bases. This: schedule will allow for Systematic 
*esonsideration of such factors as total syllabus hours, 
device-session hours,” ‘estimated. number of: students,’ number _ 
of devices available at a given time, ete. Once the ae: 
-pertinent data has been input, the System should help. , 
develop a time line for progressing through the training : 
_ program. (time-based Syllabus layout) end a systematized way 
of ited tee a daily ess Schedule ‘based: on a specific, — 
Got eagare™ ~ 3 
+. et : 
&, *, The ‘What if.cin capability will: be an. éseentiat part of 
the design of. .this ‘program. This will allow the | 
instructional development team. ‘to project the time, cost, 
_ personnel. needs,-ete.,; of .a-wide.. range of situations. © . 
. relating to. ‘development. of. the | -ordinal. syllabi. and. ‘class. 
oS séhedules.. For = instance.,’ the’ program dhoutd. identify.. 
periods. of. peak resource utilization. The team.could @liso 
ask’ the program to project the effect of.one. trainer being» 
"down" for two weeks for repair dursing various units of 


ra ‘instruction, how much time would be lost, what: alternate. es 
“- - Ordinal syllabus could students follow , ete, : In this Way, co . 
', the least disruptive time could be allotted for routine io 


| Pepairs of trainers and. so forth. 


re The program should also aesice in ‘preperation of the 
hard copy printed versions of syllabi to be used by 
developers, students, instructors, and managers and of. other. 
"o ocuments: Such’ as ordinal - ee of goes eeci ves: eben se A i 


a 


T eaenin ‘Su port. Re uireméests Pro} ram: “(TSRP). The Training | 
support Requirements Program. assists the instructional... -. 
developers, management personnel, and those who will- 7 . 
‘implement the final courses in petee ene and meneeuranene 


resource Fequsy ements data. 


ENC Cee lh ahha Be es 


Ad os reeks en : 
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. Much of the data needed coe: thas: payee can: ‘be Supplied” 
‘automatically from previously input’ programs. - 
GPAP,OHP,MSP,SDP). Analysis of data ftom. these - programs -— 
will permit astimates of. needed ‘personnel, eqtii pment, “sty 
facilities, and: services for completion of design. and _ 
development of the course.materials. The program should cue 
the user to colléct additional data -such.as student « 
throughput: data, etc.) affecting: the resource requirements 
for implementation. and ongoing instruction. ; Similar data 
and estimates should be-ifiput regarding continuous 
evaluation, -revision, and updating of the materials. over 
‘time. Using ‘all this data the. program should “assist | ota! 2 
planners and managers. in determining possi allocations of 
total resources for completion” ee ered of the*®: 
project throughout. all Reeen ine. phases of ISD. co 


The TSRP- enous. organize and structure. the data input ‘in 
such ways as to best facilitate. the needed analyses. For 
instance, the program should create or structure basic , 
resource requirements tabtes from the. data input. On-line 
review/comment and edit/update capabilities are essential to 
ensure that data is kept current and accurate. "What if..." 
manipulations and projections are'critical to this program -v 
so. ‘the. effects of various media mix ‘selections and other: . — 
resource: “allocations«can be predicted and the. resource 
requirement tables antered according ty = 


- 


The final output of thé: TSRP- isa formal: report 


Suggesting alternative course designs with alternative media , - 


mixes. and predictions ‘of ‘the total. resources required for . 
‘each ‘alternative! This is used as a guide. in ‘the remaining. 


‘phases of> ISD oo project can select reasonable 
‘alternatives. arid 11° not collapse at it eeaoe for* Tack ‘of. 


resources. oe ei . Be ase ~ 


ws 


‘Lesson Specification Program (LSP).:: The Lesson 


Specification Program will assist the subject matter experts. 
and/or other instructional developers in. developing the: 
specifications used later by authors as a design guide ‘in 
authoring student materials. 


= ~ 


The LSP will consist or on-line, interactive, branching .- 
protocols. which. organize | and.. structure the development of 
lesson. ‘specification documents ‘and guide the developer:-in 
selecting -and> developing ‘instructional components and 
strategies appropriate for each ‘objective. - The specific 
components and strategies requested by the program will 
depend upon the behavior level (use or remember) and the 
content classification of each objective. “The following 
components. will be necessary for most objectives: (Cas called 
out in =ne° FAISD model): 4 


- 


Slo Hey ie a 
4 a ars ie a 


ee a oe 


o 


‘ 
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’ c 


ae esaotece objective epeheviers. conditions, and. 
standards). ; . ; “io. 8 


2. Generality. te ; te ae 
Be Description of the instructional help. . 
4. Description of the type and number of. examples - 


5. Description of the type. and number of BUR eTEGe and 
“test items. “ oS 


6.. Description of any. special teaching points. - 


ao Te Description of graphic requirements. 
ae & 4 


Others may. be included -as needed. -To make the éyeuai 


pee generalizable, provisions could be made for user 


inition of all instructional components. In that case 


on-line protocols for Carotene and Seee eee te: Sucre aeP ee! 3 


would be provided. 


Through appropriate cues, prompts, and ‘the onaitne 


“advice or help function the developer could then be’ assisted” 
‘in correctly writing each component: ‘An example of sucha. 


component would be an appropriate, complete, and technically 
correct generality to be used 1)° by students,as the.core — 
information. to be earned and 2) as major input. into. the. | 


. development: of: NATOPS, - flight and. technical manuals.” ‘The. 
specifications writer. ‘gould be. further assisted in making 


appropriate decisions concerning | @) the type of. 


-.-. instruetional hélp needed, b) the number of ox amples. 


practice and test items. ‘equired, 2) the. Specific example 


_format and strategy to be used, and d) the practice/test 


format and Strategy, etc. Once such decisions were made, 


they. would be translated into an easily understood: format 


‘for later use_as ears een authors. when peeee Tne: the» 


final materials. ~ ae - eF 


The program "would structure nd track the entire lesson 


specification development process. to help assure > 


- implementation of proper review cycles necessary for 


adequate quality control: and ‘accountability. ' The program 
would .keep Status information 'on each objective. being 

developed. Et t ‘would also produte status reports indicating 
what portion ‘Of the lesson specification (LS) was completed, 


- who had developed each portion of the LS, who had reviewed. 
_.itoior. was responsible for its review and approval, what was. 
-the time taken for each stage of: development and review, — 
ones a 


aa é = 
ERIC 


Pic 
wt 
ri 
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To..ensure that the lessons to be developed are. 
instructionally sound,: lesson specifications. must be 
reviewed at several points by instructional experts. To 
ensure that content is accurate’ and current, subject matter 
experts must also review all: lesson specifications.. 
Therefore the program will require:a full range of 
review/comment and cross-referencing functions, renumbering, 
etce., to assure’quick and-.easy OES nee eee som of auetsey 
control functions and updates. . 


-The. program will eventually: produce (both Saat ine and in 
hardcopy form) complete lesson specification: documents ‘in a 
standardized format. The program should also produce 7 Pa 
formatted reports Af prespecified data base subsets. For . ne 
ins tance, it shoufld be .capable of. producing. a summary of is i 
each@iesson, showing lesson map, segment. titles and 
reférence numbers, objectives and generalities for each 
segment: within the: pre eons: media sal or een gaat etc. , 


Im (enentacion Plan Program (IPP). The Tapiemeneation Plan 
Program will assist project planners, managers,-and those ° 
who will in the future implement and. evaluate the 
‘instructional program in.developing an implementation plan. 
The plan is.used for coordinating resources and .estabdlishes.: 
the procedures to be applied in providing necessary training . 
for instructors and instructional managers. The-plan is. as 
also used for conducting” ‘instruction and -evaluating a 
‘students; for: managing facilities; equipment, materials — and. 
personnel; for scheduling;. for integrating the new course - 
with existing courses; and for coordina tine implementation: 


wie quality control efforts: — =. ac ne Sut? 


- 


Much of the: data needed by, ‘his program ean be provided 
automatically from previously. input data bases. ‘such as 
TSRP). the. IPP should cue and advise. the user on collection 
of other pertinent data. | The .program will provide 
review/comment and update capabilities so the data may Se 
constantly uUpéeted. as ENE le as pops mentation date — 
closer...’ : — a a, Sot 

: ‘The program Weeds what. af. manipil ations: so ‘alternative 
implementation. Pits can be somperees oe ee . 


Word processing capabilities will be needed in ‘order to 
‘produce’ initial, Speers: “and” Sass sap Lemont selon: pian 
TEPSERS « o i, ; ; 


Quality Control. Program (QCP). The- Quality Control. Program 


will assist the user in developing a report describing in 

detail a plan for complete and continuous assessment of the . 
effectiveness, efficiency,. and’. palatability of the...) ot ee 
instductional materials and management. system: -- a pe SS 
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Through the - use of, interactive protocols ‘the program 
will help the user design ‘the plan to be used for formative 
evaluation, “inéluding small group tryout procedures, 


determination of methods of measurement and variables to be 


measured, and identification of review points and 
procedures... These will be extended to include ongoing 
quality control during. full implementation of validated 
training. Flowcharts an eee of the plan 
will be needed. _ ; me ~ 


-. The program. will seo assist in structuring ext rnal. 


‘quality control procedures, which will ensure that training 


se 


ERIC . - 


Development. Phase: Programs | oe _ * a = 4 a5 


meets, and. continues to meet, Oper se pegu tenets for 
skilled personnel. _ ' 
4, ° P > 
Sample data collection forms will be stored for use in 
the. quality control process. On-line review and edit 
capability will enable users to modify forms.as neseeeaty to 


meet the ngeds. of: the instructional ae: - 


The program will provide a job: aid in ‘the. form of.a . 
checklist to assist users. in collecting all data, doing. the 
appropriate analyses, and developing the final Quality 
Control Plan report. - The system will then assist. in the 
production of the hardcopy of this report by Parnes . 

on-line an ery review/comment, edit Eels: : tg, & 


The program will assist in “the development of a. Scatiey 


-gontrol manual based on the quality control plan, which can 


be .printed in hard copy and distributed to appropriate 
personnel. Through use of the on-line. review and edit. 
capability, the manual will be. Epoeved: as needed. 
The QcP Aste basé will. serve as ‘input to the Lesson 
Tryout Program, which will actually collect and analyze 


' student performance data passe on QcP guidelines. 


The QcP utilizes input for the Problem Analysis data. 


base and the Master eia0 Program data ‘base. es 


"Lesson. Authoring ere ee The Lesson Authoring” 


Program will guide and track 1),;development -o “prototype | 
‘versions of. lessons to be used in small group tryouts, » 2): 


' lesson. revisions on the basis of such tryout ea and 3). 


Eeedyins of ‘the final: version ac production. 


Using the data in. ‘the aeesan- specification and ‘tactics 


package programs and the media format guide as blueprints or 


as direct input (as. in the case of objectives), the -LAP* 


would . Buide. the lesson: authors in developing the specified 


. x 


me < = 


WAVTRAEQUIPCEN 7T=C- -0018- i, 


aot 


instructional helps, examples, Spaceuces eect otede, -and 
other instructional comporents for a paper-a and=pencil 
version suitable for initial small group tryouts. be 
imitial version could be input directly on-line or 
handwritten and then entered on-line. The program would 
track the: complete authoring process, including all review 
and a a i aaa ? ve 


The program would aise" ‘organize and structure the 
collection and analysis of daf during the initial tryout of 
the prototypes, assisting the developers in deciding what - 
revisions are necessary. Using the review/comment and. edit 
functions, necessary revisions co d be made to the 
prototypes and they could be prépa red for entry- into the. 
final. production cycle. oe 

‘sy ‘ 1-8 
The final. output of the LAP is > printed versifn of 

Student: lessons suitable for final production in the. various 
média previously determined to be most. appropriate. These’, 
printed, versions must contain adequate descriptions of and 
references to all graphics: ‘Ccharts, drawings, Photographs, 
Slides, film, etc.) to allow the Br apuies poosveetan crew to 
enoeule the desired. end Proeueis 

Differences in media imply differences in end products. 
(For example,” ‘a slide/tape lesson requires an ee eee 
set of slides, and a worksheet. On the other hand, . 
trainer exercise demands an instructor guide, a eu sHeee: 
and: an evaluation checklist.) The printed form of the 
authored version must be formatted in such a way that final: 
production. into these different end-products..will be mogt 
efficiently carried out. Thais implies that different. ° 
formats will‘be needed. depending on the end product (e-g.-, 
authored workbooks: going. to a printer. will, be. ‘ormatted 
differently than: authored slide/tape worksheets, or authored , 
Storyboards for’ videotapes, or’. authored computer~assisted ss 
instructor. lessons). All necessary technical information: 
related té6. the media to be used must be. inelLuded- in the 
authored lesson. For example, for a slide/tape- this means :. 
How much lead time is necessary on the tape? IS a focusing 
frame’ necessary? - What background music or sound effects are. 
required? What about automatic advance: tones, -€tec. ine LAP’ 
will have guided and assisted the developers. and: -production 
personnel in making these, decisions Pe appropriate 
interactive branching protocols. a ee a ee 


« * 7 


ecson Production Program (LPP). The ‘Lesson Prédietion 


Program will receive input in the form of* prototype lessons 
from the Lesson Authoring Program (LAP). The LPP will. - 
format printed portions ‘of lessons and worksheets for. gigae? 
paste-up, -provide working copies of scripts for. production 
efforts in- audiovisual media, and format evaluation 


. 


e 
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. , 2 ‘a ‘ 7 
ee en ae instructor guides, and other. piecés of: ~ sg 


instruction. In addition, tracking and’ Ser erer ans. ° of the 
production process will be Sone @ : 


A key feature of this program will be the capability to 
review printed lessons*‘on=-line and insert necessary 


‘production codes for Eyee font and formatting. 
considerations. When linked to eompatible typesetting 


equipment, it will be possibile to print-—out 
production-quality materials ready for. paste-up or typeset 
in. hard copy. It will also be ‘possible to review and edit. 
text and production codes as. necessary on the Sy oS 
accomplish revisions-as Specified in the Lesson Tryout ° 
Program (LTP). - 


¥ 


Where jeseons incorporate ether. than print media- ai tes 
tapes, videotapes, etce.),~ working copies of scripts will be 
\ formatted and printed out in hard copy for. production - 
personnel. Worksheets, evaluation checklists, and 
"instructor guides can be produced like printed lessons to 

faccompany nonprinted | Materials. Review and edit capability 
“will also be. “available for all these items, ineluding 
Seri pes. + 


The eritire wmoduction process will be-.tracked and 


‘structured to" Provide. information on. lesson production) 


progress and revision and staffing of the production effort... 


+ In this way, maximum use of personnel tand material _ . 
production resources wil be realized. tog ee rr 


oa ae 


_ Lesson tr out. Program (LTP). The Lesson aivode Program will | 


consist of t relatively ‘independent: sections. The. first 


- will. asSist the usersin developing a°’specific plan for 


lesson tryouts consistent~with the quality. control ‘plan... 


The second. will. “provide for -input and analysis’ of reviewer 


comments and student performance and .attitudinal data 
obtained during’ the tryouts. Output will consist. of 


specifications for revision “OF materials’ based on SEyOuS * 
data. - o s ae : ba . 

The first “Section of ‘the LTP ae ‘access the Lesson 
Production Program and Quality Control -Plan'data and assist 
the user’in developing a tryout: plan through interactive .. 
protocols.- The plan will ‘$pecify the extent, and nature of 
individual and..small-group tryouts,.including a 


.lesson-by-lesson time line. On-line update capability will. 


enable the usér to revise the tryout plan_as necessary in 


mecorgancs mach ne actual . . Production ‘cyeles ee . 
: a } oF > 7 
\ : me fl 
= 4 ° ~ . 
- = == a i ‘ ? is 
ees 


a SO eg FO ie es s, ea 
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_ A ‘gartety. ee s Bgested datbteoriection’ forms will ‘a1 s6 
-. Be provided as output. -These Will conSist of student 
~ attitude survey. review comment..forms, and: additional forms 
"asS'néeded.. Where possible, the forms will be machine 
readable. for expeditious input of: data Ce. Bes ney rertese 


". The ee section. of the LTP will provide for entry of 
all tryout data. including attitudinal data, performance 
data, ‘and reviewer comments. Analyses of ‘the data will be. 
performed on-line aoe Pe evaae several. Outputs such- as: 


So We Item: analysis. indicating ‘student - ‘performance. on test 
- items and resulting mastery of gbjectives. 


2.- Student attitudes Fowand the materials by objective. 


3. Reviewer comments on materials by: objective. . 

The pesult= of” the analyses performed will. be used ‘to* 
assist the user in formulating revision specificatiors | 
on-line. Interactive protocols based on the analyses will’ 
“guide the process, objective-by-objective. Revision 
specifications will then be cross-referenced to Lesson a 
. ‘Specification, Authoring, and Production ProgramS where they 
_can-be implemented. The formative analyses and-revision — 


“specifications will also be organized using che Structured 


. report. format Sapevaeety and. will be printed. out. in neve. 
copy. . F ; ot 
-J 


BY 


| Implementation Phase” Programs: 


 Computer-Mana ae Instruction Pro ram(s) Comtp). A variety. . 
‘of CMIPs could be developed to meet the needs of the - : 

- particular instructional. system... They would. inelude 
“combinations: of the following capabilities: 


4. A variety of tests (pre, post, entry, and progress) - 


could be administered on-line and scored 
automatically. If tests were’ n6t administered 
‘'-oneline, paper-and-pencil tests could be machine _ 
Sr. scanned, entered, and scored by- using, ees a 
_#&- machine_ ‘readable’ answér sheets. . | ge 


"97 etudene: performance data. could. provide the basis for 


are ' eriterion-referenced diagnosis of strengths and 
2 2 ie deficiencies. ‘Making: up student feedback. — : 


a 3... “students could be given prescriptions ‘for a ‘ 
ae J.) remediation. or fur ther Study ieee on ‘diagnosis. 


™~ 


- 
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4” Complete ‘student records coeid be maintained in “real 
= ‘ time, providing accurate and instantaneous hard: Copy. 
<. printout of all or: ‘part. of ‘an individual peek 

, “profiles z 


6S. : Banksfof - ‘test items could be maintained ae 
pe to provide random alternate forms of. 
_ objective-based ‘Student, lay ; 


AS student. ope ares asta ould: be automatically 
eas '. combined to create large data. bases, which. could. 
= 2 - provide normative references. and indications of 
: instructional materials effectiveness... . a 
as, Personnel. and. physical resources Ould be scheduled : 
- and allocated. automatically in accordance. with | TS, 
‘Student progress: andthe instructional a . 


characteristics Ce aaa 2 


A fully- interactive CMIP. will possess all of - the above’ 
Capabilities. It is conceivable, however, that the os 
instructional program may not be- conducive to or ‘Fequire - _ 
such an array of: ‘Capabilities. | i a | ge 

ore , os “ , 

Com ater-A’ssisted Instruction Pro ram(s) (CAIP).. As with _ ele 
e aw E capabilities is possible for 

- incorporation into a Computér-Assisted Instruction frogram.. |" ¢7 
At least the Fol towing Should. ‘be eae table Se ace 


~~ 


Ord, Wl eeeane shoula: be eutnored ‘and. edited on-line: with a 
Mee? a versatile, yet. ‘easy-to-uséd authoring: language. 
ae arr _.- .WitHin the confines: of this authoring environment: it. 7 
- Should be possible’ to. separate. course "data" ‘from ee 
course. "logic". for: economy. of authoring. iarge , 
|. amounts of lesson material, and it Should: be 
“+ possible to define both ‘learner and program . 
' - controlled strategies, including simulation and . 
tat modeling activities. . a . 
2. Aetual instruction ‘will ‘be ddministered eee. by. at 
“  " system of remote student. operated ‘erminals. 4 
yo _3e Student performance data will be: collected ’ ‘and. . 
ne a a -compiled for purposes of assessing individual... °°. 
_ ~*~ + - progress. and. evaluating | instructional materials. 


7 adaptive instruction with: exte@five individualized 
-’. diagnosis and prescription. and high :degree of 

_ learner. control Supported by. ready Student ‘access to 
-. “ -eritical “performance and curriculum parameters. and. 
7 ae ae de) functions to pea gd prmsteey Bultaiee.., : 


Noe ae RT ere 
meee, 8, Gipee eG aa UN ee: pe 


A. The: program will be capable ceGivecinatt highly 


° 


ef 
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roe a he variety of: report. formats wilt be airy. 
"<, aeeessible for both ind: idual and group 
performances _ se , oe 4 ro 


“6.: Student] records will be- maintained on-line. and eS 


eae: r 


“ 


‘An impressi##e: Raaete ioh..of media eee be: Linked: to. 
-the basic CAI hardware, bub.w uld be limited by cost) 


‘considerations and :media requirements. -In addition, several 


' different. ‘student response modes. could be made available’ 
(light pens, touch Sensors,: keyboards). a Ces 


Le would. -be- possibie to- provide for complex. simulation 
aad gaming. exercises. through. a CAIP.. These. could: either: be 


designed: ‘for individual simulation. environments or. for those 


allowing anteraction of several ‘students ‘ever. the ‘system, 
with structure. provided by. the: program as needed. 2 


3 Programs: ‘Peripheral to. Mainstream ISD Activities’ 
- Historical: Trainer: Data Pro ram CHTDP). -The. Historical 


‘Trainer Data. Program assists users “in. collecting. and . 
‘organizing: ‘pertinent: data “concerning - -previously: used: 


trainers SO be eeeauied with curtent : needs. can: be sedan 


the: HTDP. “should. cue. ‘and advise: the: ‘user- in. “the. 
“goBlection. of. ‘the. necessary: data, | such: as oes 
_& ~ te 


cle, Characteristics: of available trainer devices. — 7 : 


Muah “pemesistrated mrrectivenass of devices in training of 
5 ee ee objective “ypess ee oe 


woe Cost, maintenance y and repesr considerations. | 


bas The HTDP ‘should ‘structure. the. data. input. and 
automatically provide some predefined | comparisons ‘between 


“The. ‘ATDP ‘should. output: the. predefined and: ‘unique “200 Ue | 


” comparisons ; ae some. easily. understood format. The program - 


‘should: ‘also help format and generate (automatically, 
_ whenever, possible) the. formatted report and its updates 


relating | to historical trainer data. es 


The. HTDP must allow: on-line review/comment and - contaguat 


ees The data: input and output by the HTDP> ‘should b 
_ referenced» for use.in later: programs such. as. ‘the: Trainer. 
“Requirements Determination ATRD) and the: ilar beeniose of 
-Wrainers Seon am. (PTP). _ _ ee 


~* 


—_ 


Enc La Sie ea per ope 


wee. 


devices.. The program should also allow for specification of © 
F lek a: conpentsons ane for. "what if...?” Manipulations.” °° <a 


eee 
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Trainer Requirements Program (TRP).: .The Trainer — Be 
. Requirements Program will. guide the user through. a process - 
- Of collecting and analyzing ‘the data needed to determine the | 
‘types of trainer devices best-suited'to the training . ant" 
requirements of the specific program being developed. = =” 


The program will provide automatic input. from previously | 
data bases. and/or cue the user in collecting such data. 


of. “Types of objectives identified as "device" session -* 
_, objectives Cautomatic input). it Tae 


oe en Hardware/software system capability requirements of = 

a > 6 @ach "device" session objective [according to the | 
\ objective, what hardware or software does the = ; 
a : student need ro manipulate (i.e.5- radar :-System ys..." 


att * GAs of : Btn 2G j ; ' ae = ; i * 4 . - ‘< 
cop 0. /Aas- Proportion of hands-on training time to total ™~ 02! 
ae i e aes co Syllabus: length. bi a oe ore: a © ae 


36: Hands-on ‘objectives time ‘requirements.- 


; es Mee te 
ee 5." +Enstructicral presentation requirements for each | the 
“trainer hardware/software Systems requiremeht-. ©... 
. ‘@ategory Ci.e., display requirements such as audio, ..-: 
_ -Line-drawing,* visual motion, etc; student response ~ 
“requirements, .such as: written or verbal;,. point, © 2:. | 
* manipulate hardware, ete. ;: response détection and... 
evaluation requirements, feedback requirements)... - 


. Other data.is collected about ‘available or previously 
. sed trainer, devices, such as: | wee ws 


- te: 1. Sensory output capabilities of the. device (i.e., how 
Paes 7 does it compare to hardware/software requirements of __ 


; objectives, motion.simulation,. amount of motion,.: 
a. etc.).6 °- a oe ake ge Gee re te ote 


o/. 2, Capabilities of the dévice to.meet_instructional.- 
ie. presentation. requirements. a ee ; Oa a 4 
3. Student evaluation data on use of each device in 
préviouS’ training settings. . (Did. students master... 


“similar objectives on the device?) —~.~ LN yy le 


oe 4. gfomparative cost and’maintenance data. - 


Other budget and time restriction data. is gathered, about 


the project itself. All.of the collected data must be. . 
compared. and analyzed in order ‘to select the combination of 


. a ~ te ae - cee ome 


ee ot eee ee ene aes 


: sevice which will best: meet.” ‘che instructional objective’ 


+ 


implementation, “ete. The goal. is to use the least - - 


eee least. expensive device ‘suiting the purpose. = 


1 


"Once such. data is ‘collected, ‘the: ‘program will perform 
: analyses, sorts, and data consolidations, and Structure the. 
. Gata.-in such ways as to. facilitate the. necessary 
. comparisons. - As the ‘analysis involves numeroug. variables, 
it is imperative that the program allow "what if...?" . 
manipuletions soia range of sha ee Solutions can ‘be. 
explored, if eee Oe 


& 


he program needs” to have. cestane: review/comment ; arid 

capabilities, » ECS final output. is a structured. 

which. makes - “pecommendations) as.to.the mix of” ‘deVices 

pest sWBte tothe project énd.the/number of: each ‘such © 

- device. equired.. - This report. ALSO. estimates. ‘costs -and sets 

ean. ecific requirements for. each device which cen be used . 

ee OF By a" ‘manufacturer potentially. producing. Such’a device. The. 

.* program. should ‘assist ‘the user in developing: this ‘report 

. 7. with -the. basic text editing and menipulerten fae of 
Bs the: vee 


“ requirements as well as~constraints: on: time, budget, . >. — 


” The system ekeure showearehecence this. program with r oe : 


oh alt programs” so, ‘each. objective may. be- identified by the 
-; Selected trainer Gey tere ine ot oe a 
*"prociirenent of mheihers eo (pte). The Procurement. of 
Trainers Program assists project: ‘planners and managers”  with- 


re the modification of exe stene.- ernst es 


F ol came 


“The program should bitomacically, supply pereisene date a 


from previously input. programs (HTDP, TRP) and cue and. 
Be kG users ihn collection of any additiorial data. The 


- program should assist: users in ‘defining the final 


~ yspecifications for trainer: devices: and procurement packages. 2 ee 
oe 


|The. program should .allow. for. comparison of.-incoming 
proposals and provide..a- scoring. system for ‘proposal 


evaluation... "What if'x..?" manipulations. should be available % 


as part of: the evaluation REQERSS - 


: <The ‘program. Reais on-1ERE ea! onde updabe®, 
‘ capabilities.” Its final toutput SS the Trainer. Procuremént 
Package which will-be ‘sent out. ge - -bids. This should be. 


Supported. by the. system" Ss text manipulation ‘and eonrens © 


Pe aaa poy he eS Be by 
eae - ; 
«) pes 

ERIC ~ 420. 4S 


‘"'eontracting ‘for the development of new ‘trainer equipment or “ 


ee eae Ny ae ee a ne eS 
2 | . 7 ‘ 2s : : A - : a oo toa " ; ; wi | 
ae ee ee ee (TPP). “The aes Pcie Program - 
- provides for. Steucrure and organization of the tactics | 

ee i ae er eee i 


ae 


of, 


~If the tactics paeetee nee, been. peeviousry ceveteses or. 
“igs. ‘being developed by another group, the TPP should permit. 
users to ‘e€asily locate those parts of the tactics package 
which are pertinent ‘to the instruction presently under — 7 
development. The Program should catalog or index -the entire 7 
_.taeties. package. If possible, | the data base should also’ 
(contain: synopses of material in. the tactics. Beka e eto 3 
tr. the tactics ‘pectage: "is being developed- in conjunction 
. with the eee ae development, the TPP should prov 
historical data ba to assist users in eres cine the tactics 
'. package for the-weapon system: in question. -It. would then 
“provide for on-line- input, review/comment, and. editing of 
the tactics package. It would cross-reference data in. the 
tactics package, with tasks listed in: the Task Listing and 
objectives eppearing in the Cbjectives Hierarchies. Thus 
amy change made in the tactics package could be reflected. in 
*.: the instructional. materials and _vice'verse. “In this nat ass 
vy. eonflicts. between procedures taught in training and - - 
“|. procedures: appearing. in the: tactics. package,. as* well. as. 
' duplication of efforts, could: be: avoided: : The: program could . 
. also assist in Producing. the nerdcony, version (of le heeded 
peckage. © . es oe ef a Be aS 
“Master Plan See PP). The. Meter Plan Program’ wild 
- aSsist the user in developing a long-range. management ed a 
. for ‘the articular. Isp. project. ‘to be under teken. a SS 


‘The MPP- will’ automatically receive. Critical data input” 
fie: the © Problem Analysis Program (PAP). -It wildLicue: the | 
., user in the. collection of additionel categories of 
- information (such as definitions of activities to be“ 
“performed, resource requirements for each activity, 
-eritieality of.’each activity regarding completion of.. 
“Succeeding. activities, -ete.).and: supply advice “as: to: 
‘possible’ sources of such information. . The. program will: 
- . ‘Organize and:-struecture the data input. So. as to best 
a facilitate. the needed. comparisons and. analyses... The program . 
- will reed on-line. review/comment. and. edit. capabilities to - 7 
allow for ‘periodic update of. the’ master pee to reflect, the. 
most. current information.. - oe ee 
Outputs: of ‘the MPP: ieetune prS ject: time. iaeece a. List of 
; progect assignments (personnel requirements), and a project ~ 
_. .PERT chert. The program will ‘assist.the user in the on-line |. 
Sa toerene of these ee Pee: and must. allow elaine eee e 


eo . 
wort 


a MANTRAS OUTEEEY TT —C-COWB=12 


"What if. inee i eadpuretions 2 Tne "What 2f...77 += - 
manipulations will allow the project ‘planners to consider Ne 
'.g11 possible variations in allocation of resourcés, to 
explore. the effects of adding or losing. personnel, 
shortening: or lengthening -the time line, adding. new goals to. 
the. project or leaving out some possible goals, and ‘to 
‘predict consequerices: of. these actions. to management plans. — 


Another output of the Master Plan “Program: is a Final 

' report which -formally presents this plan. The MPP will, | 

+ assist in the structuring, development, editing, and --- re 
printing of this document. and its updates by-using the. text 
editing and manipulation capabilities Of the system: _ 
Previously eonerscer reports. Should be available. as, 
‘references. _ 


ee 


- Precurement Package Progrem: Ts ‘The. Procurement ‘Package : 
rogram will assist the project planners in determining what 
' portion of ‘the proposed ISD activity needs to be contracted | 
to outside. groups. It will also assist in securing the. . 
“necessary: proposals for such. goods and Services tan in 
evaluating oe Cue g propecia... = eo 


ca 


_ The: program ‘wilt cue: the. users. in ‘collecting: the: ee 


- €some of ‘which: can. be automatically. supplied from earlier, ae a. 


eee necessary® for 1) evaluating. internat: capabilities 
for successful completion of given aspects of the ‘ISD effort) 
‘and 2) determining those goods and-services: which treed: .to be. 
~eontracted. A’ job aid in. the form of.a checklist.would §- 
“guide “the. user: in collecting. such data, securing and. re 
out the. proper forms, and: developing: ‘clear, precise, and... 
- “eomplete Requests for. id calacae Se which define: needed * 
eae and Services. _" . _ ae ee 

: co ee eS 
- 4 The PPP will “assist the users -in establishing a scoring. 
system for | ‘proposals’ received and aid them in making the 
oe necessary comparisons and. evaluations Of: proposals. The . 


program will- need to..allow for- defined. "What: Of eee & ne ee 


" manipulations. ‘to compare complex options, ‘such as hiFing. and 
“training new. Staff internally, with contracting. for such ~ a: 
.. services from outside, or: eve Matane various combinations of oe 
| proposals received.” - ; : pee ; 
 The- program ei alee offer Seegeue in the ‘Save éomenes 
‘and - production of all doctments (such as’ RFP's and. pee 
-.stating. final recommendations) needed for’ the. Procurement... 
Package. The structured report capability of the system can 
be used to autometically generate the large por tone OF. such 
documents . which are serespecitied and. pene eda 


o 


, 
ae 


a @ ” 7 . . 7 . a Se =. 3," 7 . - 
BG te Bete ct” og ge 
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Training Plan: Progrem (NTPP). The aioe. Training Plan 


Can 


The NTPP wilt automatically supply. peneiene: date a 


- previously input programs (such as the Training Program 


= activities may occur and what. facilities: are: needed; Navy: 


“personnel to be involved and their. ‘relationship te outside - 


contractors; time-and place of lesson implementation). Trey ~ 


-system will give advice as to possible sources of such. 


.. information... References to. help locate such sources ‘and 
' previously. developed Navy: Training Plans: should. be . 


available. 7 oe ox Ss ee oy? ee Se a a ok 


‘> % 
zy 2 . 


‘The | ‘program will: need. ‘What Lf ca” manipulation - 
capabilities for easy. examination and comparison: of all. 
possible options for resource esllocation. On-line -— 


4, Beye comment and edi t/update Capenit tiles are: also 
~ required. oy 


The program will Pree assist. eine user in writing and: 


‘formatting: any’ required . Formal documents. - Standard text — 
 @diting and ice ieee eapebel seres, wae therefore be 
a Up eens es. . — 


Pro ress: Monitoring: Sroneen. (PMP) \ the Progress; Monitoring 


Program is principally cesigned to track and report time 


spent by personnel working on each task in the ISD process. | 


it_also provides for projecting task .completion dates: ‘based 
on availability and distribution of personnel with: input |. 
fre / the: Master Pian pegeram cer data base. bn es os 


“Provisions will: be ‘made ‘for 3a: Wachine-reésdubie., 


'. ‘timesheet’; to. be’ filled out by all personnel, “which. a 
‘indicates time spent on eech task performed., Tasks wi:Tl be- 
.. coded in accordance with the TPMP for ease of tracking and. 
-Cross-referencing.... ‘Data from these sheets will- be _. 
automatically read into: and worei ted: in ene Project | 
Management CPM) data base. a ; a 


It. Will ‘be possible. to. input actual. and Sreiemoseed:. 


Boe 


reflect. such ‘things ‘as leave. requests, transfers, and. 
illness.or other unexpected absences... In this way, 2@ll. 


personnel movement. can be reflected in reports and time 
estimates, and the PPMP ‘can, serve, a necessary: es Seal ah 
eee eee ine. function. . 


eo . . » 2 : ge : Bee onl PRE 7 : : . 2s : a es 
: * . ee > ele : " eit, 8 : 


Program assists project planners in collecting ‘the eg eA 
' data-and ‘in preducing bg required Navy. aPeee ne, Plan (NTP). 
STGPOEES %. ; . - 


‘Master Plan) and. will cue users in,-collection..of additional — 
‘information (such as Navy sites where the training. = 


ie 
e 


availability. of personnel ‘into the. PM data. base. This will. 7” 


Ps) 


Se 

The user “will be able “to! use “he program tc assess Che.) 
‘impact. of hypothetical personnel assignments. on the =| 

- completion of tasks or groups of tasks specified in the MPP. 
This wid. be: done” ‘through the. “What Afe.s 2" capability.” 


. ‘The. Sroekan “will -be eapabre of providing. Hard copy. 
“report printouts which indicate tasks that are behind, on, 
or ahead of .schedule as. delineated inthe MPP. The program 


‘will also be. able to’ detect potential scheduling problems in. 


‘advance: based: on anticipated -resourte availability compared 
with task requirements. / In this way the user will be able 
- to: constantly monitor. the pulse of. the. total. TS0. effort... 


") Deseription of the Data Bases. 


a “The. user’ wid dnterace with. “he CATSDM. ee using ‘the| 
-. programs outlined in’.the previous section. Howéver, once oe 
data has been. entered into the system, it will be organized,- 
stored; and accessed: in relationship to a number of data: 

. bases. The purpose of this section is to present. a series 
"of tables outlining . the. contents ‘of each of the date bases.” 
For: each piece of data in--each data base, the ‘program used 
‘-to enter thet data will be “identified, and ee fore 

. which. access | ithe data will ‘be. listed.. ‘ wi 


Analysis Phase: Data Bases 


. PADB: - The oe ‘base for the ‘Problem reer pecs ean 
consists: of the following four major ‘data items. Statement 
of program goal(s).,- operatiorial/ development constraints 
(e. g., time/money. estimates) ,..available. or required: a 
resources,’ and student characteristics.: ‘These items. are 
input by the-PAP. - The ‘PAP data base must also contain 
built-in “information on needed data_and possible sources of. 
data. The contents: qf this data: base are used - as Opus to 
“many subsequen’. Programs/data bases. - : 


_ 


es 


. TLDe.” The data items in the Task. Listing Data ease inelude 
- statements of competency or. responsibility areas, ‘specific 


_, task -and component subtask > requirements, and task 
“-interrelationships. Some of this. information may be . 


“cobtained: from the. PADP. This. information is used in a a en 


EVS PL and OHP.. The- TLDB must also.contain. information 
necessary to explain. the task listing development. proces 
and. ai aoeliget involved. - es 


_ SELDB: The data items stored in the. ‘Student Bntrystevei- i 
‘Data Base constitute a profile of the. student group and -: 
-. inelude: preskill measures,-.eptitude scores, previous. job... 


oe 


4 duties 5” feeenee end/or pay scale. “Much of this data will? Lang 


ie > ee ae, 
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eo 0 6 re a ee) ee ee eo oe. 

: in- quantitative form. Some of the data on student: °°. 
‘characteristics, which is'part of the .PADB, may be. 6 oo 

incorporated into the SELDB. The SELDB iS accessed by TVSP, | 


ne tee 


TVSDB. The data base for the Task Validation: and Selection . _ 
Data. Base will include the various tasks provided by the, 
. | TLDB which have been ranked according to various training. 
== categories. .In othér words, the data in the TVSDB/is the ~< 
* Same as.the TLDB except that it has been validated and 
Sorted. Furthermore, data from ‘the SELDB may be used to. 
a further categorize the TVSP data... The information in the . 
| -TVSDB is.used by the OHP and MSP. ,’ Stee pF ebee” ge 


-QHDB. The -Objectives Hierarchy Data Base consists of a set 
of Objectives for each of the tasks stored:in the.TVSDB and . 
_  their“hierarchial structure. This data is used in the MSP, ~ 
+. SBP, TSRP, and LSP: 0) wt ye Fe . 
EMEDB.. The Existing Materials Evaluation Data Base -consists | 
of information indicating sources of existing | oe ao 
-" materials/equipment related to the- objectives in the OHDB. 
aes = must also contain information which can Suggest potential 
Sources. of material for the type of objectives’ in the OHDB.. 


Design Phase Data Bases. - 


at MSDB. The -Media Selection Data Base includes. data a 
'. indicating the media attributes related to each objective in ~~ 
the OHDB and the. media alternatives<derived. from these eke. a 
‘attributes. This data. would most likely be stored in table’ |. 
ed eid ha The MSDB is used. in-a number of: subsequent data 
ases. .. = oh oe | gf te Oe eee ge 4 


_SDDB. The. content of the Syllabus Development Data Base 
involves detailed outlines. of the lesson ard segment levels 
. sand the lesson interrelationships. (most likely in. the form — 
“ -" of course maps)..° The SDEB also. inc.rdes: class schedules. and 
c.1 > student timetables. . The SDDB utilizes data from the OHDR, 


“MSDB, TRDB, and-ts accessed by. the TSRP and LSP. 


nm 


_TSRDB. ‘The Training Support Requirements Data Base ‘includes 

' @ complete range of resource requirement tables indicating 

/ quantitative estimates of personnel, equipment, facilities, 

', Or services required. for the development! of courses. i 2 
also should. inelude specifications for evaluation, revision, |. 
and updating of materials. This data can be obtained. from... - 
previous ‘data bases (PADB, OHDB, SDB, SDDB) with some. °° 4 7: 

—_ elaboration/clarification. by. thé TSRP input. ~The.TSRDB is. 
*. used by the -ISDP.. ae _  * 


ee. 


// 
KO 


a: ao eee 


“LSDB. The. ‘Lesson. Specification: Data. Base Sonsises: of lesson » 3 
design. guides which include objectives, help, . examples, | pare 


ee ee ee ee a ar e. ee Te a Hey Pe agnte. b 
pos = a acer ee ao ee on : “ 7. 


- 


practice items, test items, special teaching strategies, and - 


descriptions of graphics. required. The LSDB.al’so includes: 
- status reports which indicate the status of the various 


components. of. Lessons under development. The LSDB. utilizes 


TppB. ‘The: Ppven cect Pian. Data Base: consists: OF: Lists: 


of eeeiniae/ips pucticaal procedures, ‘evaluation methods ,. 


resource allocation information, and scheduling data. Most 


of this data is drawn from other data bases. a. Bes LSDB,: 
TSRDB). ‘This data base is used. by the OCP.: =< 


cos. ‘The Quality Control Bers ‘Base. as conposed.’ of: 


» 


| Implementation Phase Data Bases - - ao . 


‘Development Phase Data Bases: a age ee 2. oe _ yma 


data as well as. the records already edéllected. These | 


; specifications can be abstracted to. constitute an eva}uation 
ee manual or pian. The REDE is used in the ‘LTP. 


"trainer use, ead ' The “LADB is used by the LPP. 


“* LPpB.. The ieeéon Production: Data ‘tase ‘Gonsists of the - : 
‘Formatted lessons and scripts’ ‘for. audiovisual media. Also., 
each: iesson will ‘indicate. its: current: production Status.” . 


t xs 


a ‘ 
‘ LTDB. ‘The contents. of the. ‘Lesson eryout Data Base ‘eonsists 


of two major items; a detailed- step-by-step ‘plan for trying 
out. the produced lesson materials and complete records of 


for ,each lesson. tested. The. LTDB will. also, contain revision 


(Specifications. This data- will Gone Pete to. the QC DB 


, 


include test items, student performance. data, learning 


‘prescriptions, : ‘and ‘personnel/physical . resources.- The. -CMIDB. 
: would be . accesséd by the SELP and. ‘the’ Scr. kg ne ‘ 


 CAIDB. A Computer-Assisted Instruetion Data Pase. gould: 
“Contain lesson text and student performance data. The CAIDE: 
_ would be cross-referenced to the mSoB OC DB, ‘and {PDB 


-reviewers" comments ‘and ‘student. performance/attitudinal gata. 


.data from the. SDDB > “MSDE, aes _ and, TPOB - It. is _gecessed by 
‘the: TSE RPS. Be i peg ge eee : ae 


specifications for. the collection and: analysis of évaluation : :” 


- LADB..” The ‘contents of the Lesson. Authoring | Datg 2 sesh ‘the 7 


‘CMIDB.° A Computer-Naneged’ Instruction. Data: ‘Base. would | es 


i el acoheacre Aa Ae eed ee os ] 


Data Bases. Peripheral. ‘to Mainstread IsD Activities 


HTDDB. The. Historical. eee Does oe "bat ae BS gee 


contains the characteristics of other available trainers, ._ 
effectiveness in. achieving specific. categories oF ~ 
instructional objectives, and cost/mainterance data. -Muéh 


of the content would be: sven e tative data. Some of. the data 


could come’ from the NTPDB . +The HTDDB is. used in the TRE. 


_TRDB?. The’ data collected. by the oraeHer "Wecuacehente’ Data 


Base consists of. detailed’ information on the ‘trainer. 


previously evaluated usefulness of trainers, and igh sneer apee 


.times. for certain trainer exercise... This data is merged - 
‘with data’ already ‘present in.the HTDDB,- to produce a 


“complete data base for each reiner.. This data base is 


-- Gross-referenced to the Eee and: SELDB’ and » accessed by the 
UTSRP, pers and EME? . gee tant 3a < *- ee 


oe - 


final specifications for trainer devices-and for trainer 
proposals. ‘This cael is pases Eon information from the. 


"s HTDDE and TRDB. 


= = 


- TPDB. The Tecriés. Package Data ‘Base. iWeigdes: intoraccien 
‘Specific to the operation/trainigg of a particular weapons ~ 
System. This data.is cross-referenced to the relevant Pees’) 


an the TLDB and objectives, in the. OHDB. 


task activities, resources needed for- each activity, ’ § - 
sequencing of activities, and. activity interdependence.” . 


Much of this data cen be extracted from the PADB.. The. MPDR o 
is accessed by the. PPP, TLF , -and QLP. . _ ; 


PPDB. The Pacele sncae ‘Package. Data Base consists of. 


_ statements of goods/services to be furnished Cin: REQ form) 


“- : along..with ‘a. scoring key for Oe and © proposals. The basis 


information on personnel -leaves’ or. transfers... Some of. the 


) 


~ « os an © oe : 
r - a goa Lo. ‘ a 


for this can “come . from. the . TSRD one oe 


« 


NTPDB. * The Navy. ‘Training Plan Data. Base contains. 


ec information. On ,existing Navy personnel, . Sites; Ss chene, . 
-.and materials which can be used, in the present development “ 


effort. The ae is’ accessed “by the TLP, EMEP, paDEa IDR: 


and TPP. 7 aoe 


e “> é : pa 


“Peps. The. Decerese Mentteriag Data re Gontains actual 


time spent on completing tasks as well. as estimated task 
completion data. Also contained in.the PMDB will be 


+data’ ‘in She: “PMDB. can ag obtained Poon the MPDB. . 


. 6 
ao. 


ew 


ERC te gg A 


u 
‘ ( ‘ - ae . . S ne Se : : a 
- - , . - é gs e ‘° = 


requirements for the tasks or objectives Stored in ‘the TLD ° 
~or .CHDB. This would include sensory limitations. of devices, : 


- PTDB ‘Thre ‘Précurement of Trainers Deed Rase’ consists of re, 


| .MPDB. - The: ‘Master Plan: Data- Base consists ‘of. gerentet ons Co 
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egos. * TABLE 1. SUMMARY OF CATSDM DATA BASES, ey 
ACCESSED By 


TLP, TLYTSp : 
aig x! SENSE ci ~ 


= Se —— o 2, BADR. oe es _ TLS. OnP ; a 


a. . = _Tvso8 P 


‘Mena | nah “surest Of marariels, equioment. . i 
, ‘ ‘ET ACES : OHOG, PADS 
Owos, TVSOB 


“ono. mss, 
_TROB 


. - . eeex meais excribures, task media sirernetives 
~ ‘ _ cutiines of tessons and segmencs, lesson 
INTETTOLETIONSNIOS, ChamR/RCUCENT ScrmeS 


: ee resources requirement tasks (personnel, ” : PADB, OHOS . sgte od 
cw : te i " equigment, facilities, specifications for ~ "8 MSOB, TRPOS . oe . as, 5 at 
: lemon ang quien imam developement seu sods. msob, * TSAP, ISO oo, ee 
; ; ono8. THOS ae < he Ee at ae: 
» 3908 ge, ta TREES “immarutionel procedures, evaluation metnoas. "  TSsROB,LSOB - acP 5 . nO 
eee? _#eecifications for dum collection, anaives . TPCs. MPDB LTP coe a 


ee : Vianney tect Manas ieraoen tieed, is S008. TP0a | . LPP . 
ey ope | specifications = +. : ye gee . : . ; ES 


ug. as lesson formats, scrrats, production status. " Laos. soos eee yp Oe ay bal 


T : lesson tryout olens, lemon tryout feecheck . acces |. we GTR, ria 
Tie Set - icamments, suggedted revimons) “student TE Ae ee , i ae eer § Seat 
a _. 1. Sertorhenca/athtude dsts_ < Bigg  - Y ee co : _ 
“ON : ae ee test items, student parformence dsta, lesrning. SELDB, EMEOB, ocr, SELP 
mo rt. pemsenetion, Rar TSROB, “SOB, - Sikes 
fete are eek hoe et * ve ° . ; 7. ae 1POs : ‘ a 
cAios os lessor! text, stUdENt reapones Cats s {LPOB. MSO.” ' Qe, EMEP aa ~s 
HTOos "*" "trainer characteristics, stfectivencas, oTeepa i lk, ‘TREO: 


‘SELDB.HTDOB OB ~ 
NTPOS 


. 


for sctrwtes, sequencing of activities 


2 ! ; Itatements of GOOGS. services to be 
o . . ; furnared, ie ici las : ae = 


“axssring Nawy resources 


_ fesources ussge. (orne. money, facilites. . : mPos : 
‘and oroyecnhons of Denonnel avadatsdity : es ‘ 7 Ay gs? 


<6 


; 4 2 7 © ba: : 
' , _ Le = Pa ” ae 
: ) . a > 
a ; ? " “4 . 
: : .: PF au 
° — 50 — _ | 
-ERIC. , oe 2 
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. “AaLsis OF: “VIS IN, Leer oF cATSDM ‘RECOMMENDATIONS ns 
“OVERVIEW. a a ° 


° 


“These - reports were generated from. the FRAMP gaat FRS 
Aircrew. Users’: Manuals, Subsystem Specs for: Aircrew and < 
Enlisted Men,., VTS. Funetional Desersption and CATSDM specs. ~ 

Bs cleared up through: discussions ‘with ‘code. i 

= Veey generous in their comments and oe 


rk ; a an ee a ae re oe a 7 
ee fact that ‘the VTS is- a _dynamic. system,’ Shee” 
which additions: and ‘revisions continue: to be a 
_ made, only: general | comparisons: can be made at this point. 
Th have: ‘been [eee as time and nesourceés 2@llowec. : 


iThere eee different: versions of: the system *. oe 
availaBle to different: ‘portions of.the service with no gne- 
-Site having. all ‘the. enhancements. available. This'report 
- will. consider the SUBer ss eet > come re all known. - 
-- capabilities. - ; ae e 


: -LCOMPARISAW OF STANDARD. PROGRAM. CAPABILITIES, 


Description of Program Capabilities : 
a ae These A mises capabilities, when available, “are? 
available edie for “areas which the Syeren covers. 


a: 


. The system was designed as a POS “gad. CMT system with. 

” extensive: administrative support and resource scheduling | 
roles, and. covers most areas of personnel. menegement and- 
resource Management of. pees nee: 


P rom ts, Aids, Advice). 


a8 ize Protocols (Menus 


be “NL 
_ The. system ; ts desi ance’ ‘with the user in imthes “Jthere are 
a variety” of menus available and. manipulation of the system 
_ should be. easy: with appropriate documentation and 3a minimum 


ae a Pee re - = they Ke 


ss 


‘Cues for data entry and handling, ‘and user assistance 
veRPeer to be adequates — - 7 no ee ‘ 
ee 2 i BRS 2 32a ~4,,,°. 8 . a 


ENC - . i: ~ . 7 : : od 52. a, 
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“Advice. - e . eis 
VTS aes chave. available an advice. function written to be 


‘accessed on .the terminal. | Information appropriate to the... 
user's location in the yer wilt help ENS pose Leiere- 


A Library. of Baa mehuads and training ey ee may. be 


implemented on the- “System but’more in the form of an 
inventory than a library. Since the basic capability is 
there the addition of cross-references may cause a large. 


data anery, problem but no programming obstaclés. . 
Error Deteétion/Reduetion Routines. : J 


“Appropriate error checking of data is covered. The very. 
mature of errors precludes absolute. recovery but much WOrK 
has gone ‘into error prevention. f 


~ 


Automatic Input. of Data. from Previous Inputs. 


The ‘date base management system available makes possible 
the excIusion’ of redundant data storage. . - 


Since VTS does’ ‘not have Instructional Systems Design nor. 
Computer Assisted. Instruction capabilities at this time, 


there is no previous data on the system from which ee draw ane. 


and some of. the need for enaF. data is reduced. 


“With the: ‘current system eapabiiities and the sdae ei oa: ‘Of 
ISD/CAI capabilities the €ransfer of data will be possible 
with a little additional programming effort. Actually the 
data transfer procedures. will probably be’ done as an me . 
inherent part of. the aoe: and CAT. Peps ts PELGS 


Structuréd Input . 2 


Data entry and aaa =: do take. advantage f structured 
input y capabilities. With specific (ADDS 2 Sy. terminal ¢ 
Awe re this capability is enhanced. With some additional: 


ee this enhanced capability ney be B2ven: to ‘some of” 


the les elaborate’ Harewenss 


ns 


Automatic tiumbering.endR oterencine. 


. Presently this is not’ possible. Some programming’ -efifort 
would , be-nééessary ito .provide this ‘ability although with.’ op 


ra 


systems configured’ “as at present, without ‘ISD. ang. GAT, Enis | 


so ican eae is ‘low ore as Bg tees ee ce 
Q ne f° * f : | ge , 2 at . > - 
! EN —" ae ee ° . 2] ee 5 oe . f 
Ty: a ae % Oo. + 


et 
oe 
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| Hierareny of Users. 


System protection. from user 


17 -C- 0018-1 aye 2 a 


S is.rather elaborately 


provided for- in. Soth the areas of sitacadea and data base. 


ad 


Oncl ine Rev teuscsuncne. a 
Comment. 


On-line review ‘and . comment, 


~ 


with appropriate: protection, 


is Standard on VTS. Changes. must be okayed but comments, may. 


be left Ecgarding any Daeeensy 
access. ae 


On-Line Update. 


a person is authoried to. 


Standard apd anaes procedures are on line under.- ‘the 


authority of eEece seco puthoriced ‘People only... 


"What Tf...2" 


The ability to simuie te. dif 
System rescGurces and =TOoject se 


hot | ‘coveres. 


~~ 


‘Communications Facility. oe 
“a3 ‘On-Jine. communications feei 
the: RSTS/E Operating system. - They “appear to cover’ ai 


necessary | ‘aspect eS 


Structured Report Format. 


pipene: “Gore student loads end 
hedules is availeble. ‘This 


concerns ee CMi- and POS. > Instructional eee is 


lities are “Syed ie ble’ der = 
the 


+ 


- 


‘ Paperwork ‘and forms necessary ‘for ‘system operations ‘are - 


provided by the System. The ca 


pabilities may be increased 


“with the ADDS 200 terminal and with some effort the: system, 


can .be made. to provide this increased capability for some ie 


the less elaborate software. 


- 


Grade Leved Coffputation 


Readin 


Text Eduting/Curriculum: Tile Ma 


ry 


7 i, 


vendor, 7 a4 3 


. a 


we. : a 
Presently this’ isnot. provided ieee VTS. The addition 
Of’ this vows. be 3a fairly stra tent. oe task. = _ 


intéponce Ca dabilitys 


ae - weet fom tae 
oy 


=” 
a, 


0 


fe oo os, ae : oe hy te 


ve. This. is possible on vTs, however, due to some’ problems... 
2 * with: protocol, it: is: not and cannot Bie" presently provided. .- 
as is part of the software pee provided by the hardware 


fond 3 
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Hard Copy” Printout. oh, 3 a . = _ 4? 3 : pe? 
The Data Base Management: System provided. gives eaener . 
elaborate capabilities in developing reports and listings © 


from the data base. Hard copy of any screen display is. 
available. 


| Progress Tracking. 7 ge ee Se 
Progress tracking in “the. context ‘Of. inetruetfonal 
systems design is not available... Student-progress is 
provided, acunaety as a major part of the instructional 
/management. - or eu 


‘Standard. Data Rase “vanagement eae 


- The standard Bpeieae: ‘ineluding QUERY (a data access 
mettoed)< expected in a Data Basé Management System are | 
| provided by the hardware vendor. and in. some ar ess, enhanced. 


COMPARISON OF FUNCTIONS FOR EACH “PROGRAM 


.a 


. nao ; . - e i 2 : 
” Pi ac : ¢ 


Course Author ing Functions 


- l) The: CAL au. Moring Language. DECAL: Ir is: part” of: 
(system: deliver d with. the hardware. although it is aici a ma 
-major. portion of. VTS. It, is not yet software connected - with a 
the VTS, -software. or. data base. Extensive text editing of. 
‘the- type. needed is‘not. yet.offered on‘ VTS... ‘On. VTS proper, 
the - following are not Reelty. available In vestue. form: ‘ 


Lesson ioeasrines 
- ,Lésson Tryout : 
_ Trainer ‘Requirements Determination = aaa es ee ee a 
2 Media ‘Selection | - a a ee re ec ee ee 
Lesson snacdfacations programs and lesson Seduction. 
programs are available by generalizing existing programs. 
_Onemypiece ata, time .the comnection may not be hard, on the 
‘whole the programming effort may: be substantial .. Oe ag 


o 


‘Functions which Create ‘Reports, 


an = 
- t 


; _. Most ‘of the oc ommenese ions in this area require ‘that 
part or. all.of the.data for the reports be input from. ; 
vexisting. data bases. In the following’ functions the data is - 
either not aviauiebie on Aes only parts si sy available.».-To. 


* 
2 


ven 


re) 
es 
4, 
( 
/ 


a : : r-6-0018-1° tt _ 
ee ins; OE 5s ae 
implement these functions it would: be necessary to write 
“programs to guide the user in inputting the data and then 
use the peg pe ee Ce eet the report. — oo. 
~ Problem Analysis 
~ Master Plan Program ie as 
‘Procurement Package 2 
Naval Training Plan «° . — : 
vi Trabher Procurement ,Package 4 bo ae 
Implementation Plan ‘Develspment(, ee ee 


ot 


. Student Entry-Level Program. _ ee eB oo we 
ce © WES. GAT 94ccehe anes on students coming into the command 
~ and will help the instructor. re-evaluate Students with:> ~.- 
.previous records, or completely evaluate any student. ~ - 
_ Appropiate data is stored to assist in planning Student ©... 
_ Schedules and Projecting the result Or the training Plans <.23 
Task Listing Program | 
ee a oy, 8 4 a ne . 
es ‘VTS Gen: cert y meet this récommendation.. Se MAS tye 
. Software which gathers and stores a list of the major tasks | 
_land subtasks. required ‘in any Dillet. Since ‘the VTS task: os 
-. . Tisting is used only for billet Selection, -it may not be:-as 
“=~ detailed as desired for courseware development. - Vggs 
oy Modification of the -existing programs would probably be 
‘sufficient. ge ON a , * 


Task Validation and Selection Program , oe 


poke . ‘ . ae - LN a : 7 . : = 7 > Py i : os o> 
_curvey evaluation programs ‘now exist for evaluation -of. 
Surveys with up to § Options per question.: © * ae 


“. In addition, ‘surveys may be generated with perhaps a. 
Slight modification of existing report generation: and text °°. 
'_ ‘manipulation. capabilities, vSince .true word: processing is we 
presently not provided, some aspects mey;be more difficult. 


_ 


°  Ob9eehives Hierarchy Development Program). er 2 ‘a 


VES does allow. and encourage hierarchical ordering of” 
objectives-on the system. “It does allow entering, -deleting. 
and updating of: objectives... aes o 2 e gee oS ae Mg 8 
Syllabus Development Program a pita, Oe 
ae JAL1 the: sof2ware “for -con tructing class’ schedules. under 

 VariSusS time and resource constraints is present on VTS. ©. 

The mechanism for automatically setting Up course “syliabd is » 

*also. available using the objective hierarchy. - °. ga ty ue 


eye 3 “Ovige 
= é 
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Tactics, Package. Program. 


VTS at “the - present ti net inelude poetics: 
infomation., Programs some considerable. effect 
vou. be ieee de ec nsiderable data. 


Training Support ee eed : a“ 


VTS. ¢an and- does. coveR, the domain of this: program. -The 
resource. configuration and. : duling software.schedules and 
monitor all resources required for = a course... In, addition, 
predictions are made. regarding resource requirements. 


Existing Materials> ‘Evaluation Program eo cr 


Existing | materials can be. located. Evaluation and ~ 
documentation oF all” eld Materials-applications +s not 
aione ; Zs eae 8 BOL -. 


“With ‘the addition of ‘some data.collection and cross 
referencing programs this may be approached, witha large | 
degree of accuracy. The programming. effort would be minimal . 
but_ ate: collection. would - be. of some ‘concern. : 


- 
~ 


. “The software for éhsostng from a. ver feey. of. eae. 
peter ade once: the ‘effectiveness, ‘and existence of those 
materials is established, is now in” “use.. so VESTS 


Ce a er Z ~~ cee 


Historical Trainer ‘Data Progrgém a ve 


| ‘VTS maintains an’ inventory or ee since equipment, status, 
and effectiveness... Information on the characteristics in. =: 
the detail. needed is: not available. In order to perform the 
desired. analysis thé data. base would have ‘to be expanded and | 
software. for evaluation of characteristics in ‘light. ea a 


training requirements would be BECESSsry. 


Progress Monitoring Program |... aes Pep ae 
As: part of. ie eyeten design this. does: not 

exist. on VTS. There are.programs in-.existence which can.- be oes 
Zenralized to produce:most of the. needed output... | ‘The’ level. - 
of. effort to produce. ‘the project monitoring. réquired-. should | oF 
9@-acceptable. “Specifications and an operation al, ee 
for ‘such a system CAMS ) have been seveloe 2, within eee . . 


. 


« . : : i F ’ 
crest é ¥ = : : Ws 


aoe 
a 


ay 
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Computer =Managed Instruction Program Lae 


‘In the area of CMI. the following 1s characteristié: 


a ey gt VTS does offer a full range of on-line tests add 
, ‘evaluation. or entry of off-line tests. 


2.°. Student. performance is saved and used for student. 
feedback diagnosis. a 4 


3:0 Student: prescriptions for chudy are five n although 
~  - Auman intervention is required. a 


hs | Student records are managed in real time. 


6 eens of test items are stored to produce random. 
“ objectives based tests. ; 


6. . Instructional materials. ere evaluated - vin ‘Light of 
student performance. ; : 
a Personnel and physical resources: are. _Scheduled | 
» BUcOmaticalty., ; : . 
: _Computer-Assisted Instruction Program : — : — 2 
a 
With - DECAL II from the. hardware vendor somé aspects of 
CAI do. become available. ~ It is, however, a substantial 
effort .to implement as'is and as is it. doesn't conform to 
“CATSDM. recommendations.. Since it .is not generally thought _ 
". Of as being included with VTS we will consider VTS without 
. DECAL: is Se ae In ena s Oc ee follow: ng becomes sPReE cnet 


as There is: no lesson ‘author ing. capability on ‘VTS. . 
‘2. ‘Actual Instruction is ‘not administered on-line 


oe . 3s Some Student test performance data 13 collected... 


4. There are a variety of. formats . Tor accessing. 
individual and greup performance data. . os 


§... Student. ‘records are maintained esas put! not’ nf 
. the detail required bas cal. PREY Gra Urea eee an 
real time..- ‘8 es 7 we Sate, 


mah Bigs > UTS does. plan to. have a CAT component: ‘and may at 
ase ~ some future time fulfill all of these _ 
aif. “recommendations, » however, DECAL ase -Ls. seen as. 
a read aausee >” as sas _ ao 


at *. 


r 
& 


a oe ae os oe ac oe ‘du, aa 
A es ee ee 
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‘Quality Control Plan 


VTS. ‘does: not assist in deweiGoind a variety of uet fey: 
control plans. It does however, provide for a. given plen 
and the sepricericn of verious plans which may be developed 
off-line. | “_ = 


COMPARISON. ‘OF DATA. BASE CONTENTS AND’ STRUCTURE 


Description of the. Datia Base. — a — = 


The fol lowing ae bases are not presently part of the. 
VTS data base and would need. new programs to be written oe a 
order to add them. Most ofthe. ‘above. fall into two 
categories, one. (#1), development of curriculum, and two 
(#2). forms: generation with the- goal of curriculum: : 


slese uae: . a ft Se 2 
Program porn Data Base’ Use Catégories 

Problem Analysis / .  PADB - + #1, #2 
Master Plan .-MPDB - _* 4.5. #2 a bs 
Procurement Package. , ©, BPPDB * we Me gg a 
Navy- Training-Plan .. *° -NTPDS — ‘#2 ees ee ae 
Lesson Interrelationships : oe ee ue ek Oe 
- -Matrix - * IMPOR ¢ fea + Pies 8 ~y 
Tacties Package +’. FPPDB 2 so * ie * .- we 
Training Support ae oe 4 5 4 a 

Requiréments |. Se ee TOROS. Ses aT, z 
Historical Trainer - Data “. “HTRDPDB -: - #1* ae”. 
‘Trainer. Requirements oe Bee eR PHS. “Shane ty 7 Ve eg ae 
“Trainer Procurement Package -TPPPDB ... ogo" Sige PE Bie 
Implementatian Plan °°. IPDBP =  .:* #1, #2 oe 
Quality Control Plan. ~ _QCDB - . #1, #2 a 
Project’ Management’ a - -PMPDB. 7 - a a 
BO am 3 = cs 


; ‘In the. cases ‘where forms: ‘are. developed: (#2), text. 
editing/curriculum file “meintenance capabilities ‘would: be 
almost a necessity for ‘computer poner ation ene uederens. of . 
these OTS a 

The remaining portions of the. data. bese exist ot? least. 
‘in part. . . 4 tee Ore . 


t a oH Se .- : ee oe oe, 


‘Syudent Entry Level, Data Base ae = ¢ oo ae 


This deta does’ exist and ‘can be ‘updated hie he - student’. 
“arrives after being initially entéred on notification. ‘of 
transfer.’ Moreover, the data is updeted, automatically . as 
students progress. - . fe ere eee ee 


Gg a ee 


re . 
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Task Listing. Data. Pes ae i. ° 
There. is alist. of ‘asks 54d subtasks contained within a. 
course hierarchy . a ; Ss . a 


_ Objectives: ‘Hierarchy’ Data Base — 


_ There is a’ nierarohy OF obsectives within the course’ 
hierarchy.’ - i. a 7 an 


Surge Syllabi Lesson: Level Sata Base. 


fee “There | ‘is a syllabus. for each of. four ‘out oF five levels 
of ‘tasks and Subtasks, within 2 hierarchy. | Bee tg. 


Course Syllabi Segment Level Data. Base: 


— 


‘This. is -also available. = ao eee 


= teatia Materials Data. Base a 


By There is a list. ‘of ‘all: existing training’ materials’ at’ * 
-* particular Site, but this is’ not connected with materials at 
“other sites’ and it is referenced ae ‘the. course level, not — 
the objective or task aoe ae ae, . te Me ae 


a _ What. presently exists. is probably’ “adequate. ai 


"Lesson. ‘Validation Data - ‘Bade. | 


A history: ‘is kept: or ‘test: Sue seior <. and thse rectors Uae. 
order to validate ‘their: effectiveness. — ‘Student histories... 
| are’ ‘further : ‘Kept -for future. digesting Of this material. 
“However, for a rigorous validation of the lesson. materials *" 
‘more detailed information 13. needed, and the, programs would ~ 
need » ‘revision. accordingly. . 4 aor oe oe 


> 


Computer=Managed, Instruction Data. Pase| 


= This date bose is completely ehera: as: Hie VTS System: has 
been ‘enhantedg specifically fOr. eae “Enhancements an scope 
"and Sophistication are needed. eS a 
eS - ots , 
© Computer-Assisted’ ‘Iistrueticn ‘Date Base | Ca, ; * 
et eee material and instraction | are. “not: ‘on-line, but 
Student performance is Kept - ‘and may be updated | on-line. 
eGrouD. vand_ eudent ‘Reports are also. generated. a: 


ve: 
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‘This addition Houvd require a major effort in 

“programming. and ‘planning, ifiit were attempted to add this 
“to VTS. .A better strategy would be to design Jinterfaces 
with. other systems dedicated. HO. -true CAT... er nae 


Le'sson. Specifications: and Text Data Base ; oo 
“Many lesson specification elements are sires part of... 
the eae base, Revet Sas more would be necessary. . 
‘a - : . 
—_ : ~ 
a 
Nm - * i : 
ot 
— **s ese | 60 | : 
MMe BA te ; 
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= vos 


| | "SECTION Iv) 2) : 
_. PROGRAM DEVELOPMENT: AND IMPLEMENTATION PLAN 
It is possible to generate-many reasonable alternative 


plans for developing and implementing @ CATSDM system, each 
with its own Particular strengths..and weaknesses. — 


If it-were diffieult to.extract clear recommendations. ~ 
‘ this study, there might be.some merit 
‘to providing some alternative plans.- However, as the 

_ recommendations of this Study are Strong and unambiguous, 


rationale. — te of 2a aa “any , ek 
“PROGRAM PRIORITIES IH LIGHT OF THE COST/BENEFITS AWALYSTS | 
“Benefits of Autoriated support to FAISD Tasks pages at 
. Many factors affect the eose to benefit’ relationships 


which imply the. priorities which should be afforded to |. 
various identifiable components of the CATSDM. system. sees 


ac phese are discussed “in Date Item.004, ISD Task. 

Priorities for CATSDM, of this contract, in the | » wa 

"Implications’ section following each.df the questions in — 
ne weber oF. that: document;! In “the interests of generating” 
../-a™stand alone" -dociment as this final report, these are iy 


a. ‘background rationale for the task selection ‘and Prioritizing. 
efforts which’ follow. These, may be considered to be a 
benefits of automating the ISD. process. a ~ 


-.¢ <Tasks which can be turned over to personnel with little. 
Or-no specialized Knowledge or Skills, frequently will be | 
benefited in terms of consistency and overall Guality | 


monitoring: and specific zuidan € and Structuring of subtasks 
.. Can’ becprovided with some effectiveness through automated, 
management Support of the task. 


control when’ closely monitored.or supervised. Close . 


_ . . Tasks which require the specialized input of a-very few. - 
-. key people are good candidates for’ computer support. - 
-. Extensive text manipuletion Support provides a production 


“+ more economical editing and rewrite capabilities. In 
addition, thé expertise of these. personnel can, in some * 

cases, ‘provide a leverage effect. The impact of their input 

can be multiplied through utilizing it to design nigh-payofrrfr 2 
- automation functions. related to the task. These can lead, ~. 

in the long run, tc increases in the effectiveness. of their” 

input. .  ~ *s Ge ae ee . ~~ s 


221 aan a cs i ns arr oe) oes 4%, j 
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~ -8nytime fhe- Ts D activity. is cee: in many 
simultaneous tasks, management complexity.is greatly °° .+ 


increased.:. Tne adventage of: computer support in tracking - 
the progress of: the-6ngoing tasks and coordinating the - 
personnel and resources tney ‘require is TOE EOS ae 3 fo 


F 


. sone of the eoseace de Heténctes of ISD ‘activities, \ 
both within and - without Department ‘of Defense, is the almost-~ 
universal lack of provision for collecting and ‘analyzing the. . 
sort.of detailed data which would provide a good planning 
base for efficient use of personnel:and resources and td 
support the extensive scheduling -needs of thé overall. “ISD 
activity. Computer support can provide the means for 
collecting the historical data base and analyzing its- 
implications.: ‘This willl allow much better plananing - and, 
scheduling -of critical: resources. such as, in this case, key 
Per egine who | cannot. easily: be. J epteced on ‘the task. 


aed 


) The ‘requirement : for: arecnsive ede of personnel 
prior.to the completion of the task decreases the degrees:of 
scheduling, freedom available to..the management of the ISD 
activity Since the training time must also be scheduled 
prior to beginning. the ‘task. -.-In general, as scheduling 
problems become more complex, the potential benefits for 
automated system support increased. 


. -To the extent ‘personnel must: “perform their ‘jobs without . 
closé supervision, the ISD effort may potentially benefit oe 
from the collection of more detailed data on performance of 
intermediate ISD subtasks and from the capability. TOs provide: 
the affected personnel with more structured job aids, : 
yithin=-task checkpoints, and feedback. This effort .can be 
zreatly facilitated through the provision of easy “access to- 
and the requirement for, frequent interaction with the: 
automated ee system by the. er teeved clipaack 


> . 
. a. 


-To” ‘the: extent that: B task, can be broken up: into, easily 
identifiable subtasks ,- computer | support of management makes | 
it ‘increasingly possible to closely monitor the progress of: 
Lndividual personnel within the task. Thi s:: provides a neue "5 
Jase which can guide management towards necessary 
estructing ‘of the task and/or toward additional training. 
reeds in regard to specific Pere on= Le 


-To the extent to which progress sheekpotdes capieasily - 
ind objectively be determined, it. is appropriate tor monitor 
ind.-report on overall task progress through computer | 
support. The benefits in terms oF management and control 
re obvious. 


wee, 
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NAVTRAEQUIPCEN 77-C-0013-1 0 
oa -External.monitoring of each task becomes more. important. 
in cases where the levels Of personnel: assigned to tne task ° 
are unable to..adequately. monitor their own progress. rat 
Automation. provides potential bi fits in terms of — 


. information and Process ‘support 3 Vantin i gee | 
: -The importance of-a real tim anagement system is .« 


greatly increased on those tasks where the timely completion 
of the task is critical to on-time Start-up and. completion. 
of other ISD tasks.. Computer Support of the manegement  . _ 
fanction and/or .implemehtation of the task offers ° ca 
" correspondingly greater potential benefits in these cases, 
-.especially in terms of early identification of problems, and 
in: terms of exploring the effects of alXernetive. - 
“solutions.” -.2:0 olf maa dg ee EO gst ie, 


- 


'-Computer-support can greatly alleviate communications 
problems which may arise between personnel from varied . cos 
specialized backgrounds by providing a central clearinghouse. 
and definition point for all terminology used, and by 
providing easy access to both formatted r€ports and detailed 
raw.data. «The computer system can greatly streamline the 
information trensfer process while providing a valuab]at% 
discipline and consistency Support Function. co Ne 


¢ 


nonstandard terminology., procedures, tecrniques, and so on, 


7 ~There may be many efficiencies effected in dividing 
/tasks into pieces which could be worked on “independently by 
many personnel. -However, the cost of this efficient use of 
' Manpower. is often management complexity. Automated: systems © 
‘Support enables you to take advantage of your full personnel -° 
pool much more efficiently. ga Fe ‘gh ee ; i 
2 -On’ long tasks, frequent detailed reporting may have a 
strong motivational effect. The primary advantage is that 
personnel‘ with access to frequent detailed. progress: renorts, 
_are-able to maintain their sense of progress. Without this 
Capability, some tasks seem to drag on forever at. Le 
considerable cost to productivity. end morale. : 


} 
i 


(cSimilarly, frequent reports may he necessary to 
maintain a sense of getting somewhere in those cases where 
the task may not be. clearly releted to the final project 
products for the personnel engaged in the task. With 
automated support of management, it iS. possible to’ maintain 
a sense of "getting. somewhere." i * te “ee, 


‘ 5: ae 
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=Ta asks involving extgemely. well- dePined oreeeduces:. are 
very good candidates for automated monitoring and reporting | 
and/or for real time cueing and prompting: in terms of- 
interactive job aids. This can result in better audit _ . 
control (through Standardized approaches and error (le 
detection) and better efficiency. (through prompting and 


“pacing). : ‘ ie 
rar . , j 


‘=-The more mechaiiical Yer sthoricemicar sy defined a task - 


is, the move ‘that task becomes a candidate for complete 
En vers: TheESey freeing staff ror OCner activities. 


” x 


-If. enere is 2 complex mix of pEecoueeren and training 
facilities required to carry- outs task, there is iné¢ressed 
probability tht the management of the task would be 
benefited through use of asset/facilities Scheduling and 
managément : ‘program support. ae - 


- In any task requiring avery large typing load, there 
is a high possibility that on-line “text editing and 
“manipulation. support. would be cost effective and would 


result in emer oved- product qustieys a eg BP * 


o 


_=-The. generation of extensive’ reports and/or 
instructional -products implies.a’ need for scheduling. Of: 
production assets and facilities for, possibly extensive text 
manipulation and/or headlining, typesetting capabilities, 
end for the creation and maintenance of data base to be. 
manipulated for “report generation purposes.  -. .. 

too ; 

-~Extens_-« -equirements for training facilities can 
generate ccmrtlex scheduling problems. In the-case peo a 
hands-on training devices, there will be a.need to edule 
Students serially without conflict to take maximum advantage 
of the usually limited supply of hands-on devices. In the 
case of group classroom requirements, there, is a need to 
schedule students in such a way that: students with the 
proper prerequisites are scheduled Simultaneously into the 
classroom-resource. Seheduling Support provides Benefits 
through more efficient - use and maintenance of expensive 


3h 


resources. . ‘ : : Ps 


* urn Pommnataea to . ee collected by project *nerSonnel can. be 
recorded and- manipulated by the automated management “system. 
In those cases where information is to be generated by the 
project personnel, “eémputer support. can help with cueing and . 
prompting programs and intera etive data input rograms in 
insuring conse seensy of format.- ‘Structure and emer Toes 


_— ae 
ry co : z Be ot . Poe 
- : S68 : 
ee : , ; 
: re = 
2 a rs t 
© PS z . . , 7 . : ta . S 
645 =28 : . 
‘- Q : . . 
ERIC « * 7 : 
ENC 67. 7 * 


re 


oa: 


NAVTRAEQUIPCEN 77=Cc-001%-1, °  - , 


ae ie Properly designed automated data collection techniques 


m5" - -SUpport is’ hard ta deny. Considerable time: ena expense‘ can 


che 4 Of this validation process. Bc! A 


“tend to enhance reliahility and completeness of information 
generated by people with wide veriations in. experience and., 

' beekground. . Be EF ce oe 8 oe ca : 

. a ae a , fo 
M elf data-is to be used in multiple alternative forms, it 
is extremely important ‘that the data base be quickly and ; 
easily manipulated. In most cases, as.the data base. becomes 
large, this is possible only with computer support. 22 “ 
. - vs id % re as td bl Q . 


- «If data to be collected has a high. proportion of simple: 
‘ ' “alphanumeric codes or repeated brief text. string such as key 
words or abbreviations, computer wos Ort can offer valuable . 
\YWata packing, sorting, retrieval, and error-checking ” | 
Support.: For those longer prose Hep te Seca eat 2 
' P@ragraph length or longer), laFge-seele Storage, high speed 
retrieval and output, and complex -téxtyediting capabilities: 
are important advantages tdibe offered by computers.’ 


Se . vce i ae a ee aan ee Po ee 
ere na ~The amportance of the ‘sophisticated error=checking and... 


a 


correction capabilities available under automated systems... 


— be ‘saved as more and- more errors are detected immediately 
{ when made, or as errors become easier and easier to correct, 


_ _- 7In cases where it would be relatively simple and 
‘inexpensive to build-in data validation processes, : 
automating this funct#én is potentially advantageous. In 
- those cases where deta velidation is somewhat more 
‘difficult, but. due to the criticality ofthe data, still | 
very important, a Separate decision must be made on whether. 
Or not to. take the extra time and effort to automate per&s 


+ * 5 7 ' ; ; 
_. w=In cases when data could be Zenerated as it is needed, 
-Pather. than Stored, often'a simple data translation program 
“Or generation slgorithm cen replace personnel or Storage .- 
files in supplying this data. f we 4 


—_ 
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ae “Often, date structures possess a high degree of 
is 9 bubltein redundancy ~ For example, it is possible ‘to 
dotument the various rationales for computer. support.of the | 
individual ISD tasks by storing one Copy. each of a set of. 
footnote comments, and then referencing them Dy. a single. 
number (as was done in data item 004. of this contract.) In 

~ . this case, whole paragraphs of data were coded and a aah 
Pala reférenced -as.one- and two-digit numbers, and then were’ - 
« @xpanded into the RParegrapns they represent by the system et. 
retrieval--or. output time. Such data translation and packing, — 
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©. gapabttieies. can “greatly: reduce’ the acne: of storage : 
_ > required ‘for large data bases with the proper reduadaney. Big at a 8 
_ characteristits.. fclibrary of student prescriptions’ in. ‘che 

CMI data base would. be a similar paeeene ee ie on Sh de 


; fo, 
> £on: data bases ofeany” ‘size at all, retrisvel by ad yora.” 
techtiiques: or retrieval. through extensive’ ee 


and. indirect accessing: techniques, is inefficient,.: .- 


~ which 


time-consuming, and expensive by manual methods. | This is: an: 


extremely. ee area of advantage for automated Syecens 


mepeo ee ei 


in any case when information, to be- retrieved from the 


project. ‘data base needs to be. formatted and/or manipulated. 
‘for reports ‘or other: documentation. | eg 


2 ee - 


The more. ‘often updates: “or tances. “to the data base 


eeeur,;’ and the. more extensive%these modifications are, the 
greater — the  ,adventage ZO be reelized by automation. of the 
aa base. © . i. 


st 


-Extensive nericsl analysts requirements on ony or. ry 


* 


parts of. the data base strongly imply that computer Supper’ 
“, would be advantageous. eee 2 .G re 


~_ 


- aTe ore’ ways information needs to be sorted. and theo: 
. evel the data base. to be’ sorted, the ae ig the eo o: 


accrues: Rae oer support. F 


Pat 


“ iNnxs between: different parts ofr "the date base. are most: 
easily established and maintained, -and most economically “ ° 


taken ‘advantage of,. through automateda systems procedures... 


In particular, merging of data bases créeted at different 


times and/or for-different! purposes, is :most easily 
accomplished” through: eross-referencing techniques which 


26218 strongly imply compurer Support.’ i, a ae 8 - 


ee. cad 


"These are representative benefits. of automating FATSD ao 
tasks. These must be considered, however’, in eee of. oe 
Goer ae oe gees 


oe 


” Cost- Related Development Pridrities 


This. gelcicn, will discuss ‘Several Geueieoneat priority 
‘factors, ‘those which are based on th 1e@ cost analysis of. the. 


PC ag Aviation Instructional’ Systems. Development. (FATSo) 
ENE. those enact are. then poueee ae ay. basec on ae 


.. . - : oe é 
as ge ee 


ee ee. | ’ Rigs eo ee ee it * oe ‘ 
a er - “8 ; Se as in ee 
ee ges 2 *, ae . akctre SytS, PRE Se ae ea 
2s if. “ a ‘i er 
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-Rubomated yet an: support Gites significant advantages” 


Par 
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7 evolving tie =e, Sp: ‘such as. work “éurrently ‘Semne. 
_ @one under Army Reseagch Institute CARI), Defense Advanead 


peoeened Pe age ed nas endf other ‘sentracts withing 


DOD. 


eee ay, task priorities. Verwety eost related: age : 
generated through ° an: analysis of FAISD tasks 
terms of manpower and resources. required. compared ‘Lo 


- 


task. 


ae 


the potential Beier its: of enn ae Support. fOr  e8en - 


This’ analysis: Was er upon cheese major Histone. oe: ‘thé | 


~automated Support. 2) the: -Ipact of sevén’ "oriticalit ; 
faetors,” and 3) the "personnel leverage factor". of the - tsk. 
The relative degree to. which -a task: could. benefit” from”. 


~ automated Support was determined on the basis of a set Gf “YR : 
information Management. concerns: Each ‘task. was. rated by the 


_ Study group-as either benefitting or not benefitting from ° 
automated support On each of- the information management © 
areas. A score was assigned (from 1 to 48.) to each task 
reflecting how many of the potential areas. ‘of benefit from 
automation eeericd to poet task. . , a 


NR as areas were. judged to ‘offer especially important . ; 
_ potenWial benefits: from: automation, if: they applied toa. - 


task. These wereSrated as being worth’ tetther 2 or -2.extra- 


poirts. each, which, were added. to. pias tofal for each: ‘task as 


_ applicable. 


Q 


ae Each’ ee was ‘then: ‘examined- to. de stermine. the: number : OF 
ee “man-hours likely? to be impacted by’ computer Support. for .the 
ae task, - ‘A "personnel leverage": Weight was assigned. to. each 
a task‘ based on ‘the number and | type of- ‘pedple affected. This: 
‘Was: considered to be very important, and as much as 15. ose 
~points~ were: assigned to.some tasks’ on this variable. “The * 
Wades a3stene were subjective, ‘but ‘represented tne” ah 
Judgements. “Semiagr scutes -and. well=qualified. ISD personnel 
With: ‘background sin pane pres e cet sens to. developing: 
“Tratrvetion? on c 7 


iv . oes 

+ Le, oe > 2 | 
ao a total. score: Was rpHieg: comp cee for. each task. wand. the. 
resulting priorities: are: cua as follows: ee 


\ 


f° 


degree to which - the particu’ar FAISD task could benefit, from 


Ses 
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‘o = 3 < ae “Priority: + _ 7 ee eo ae 


“Pro ram nithber referred: in Rig 


Programs _— atte ia ae 

: s 
Standard ‘Capabilities * 
- Lesson Specification: Program. - 7. a ee 
Lesson. Authoring. Program. ve 4 uti ne oh ee 
--Lesson, Production Program * ae tee a ee oe 
‘Lesson Tryout Program ee ee ae 
| . ¢ 

( 

( 


e a * 
gs 
UIA OU LO) 


e-e@ 


CMI Program ~. han FR, . Pe @ ie 
/ Media: Selection-Program .. ee eae we ot 
Progress Monitoring - Program ee He Wis 
~ ¥Number. in. parentheses refers. “to program a We 
_ AUBDEr nee in: Figure Ts so 3 a S48 


et a Set ne Qs 
7am ; a an ee 
ie ee 
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Priority 2 ,%2 * = - | 
oe * eS _ “s ? on 


NOW Fut a oO 
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. . Objectives ‘Hierarchy Development ‘Program. Fee BD. 

- CAT Linkage Specifications ‘Program. tee oe a Oe ae 
Student Entry Level Program a mC By te 

‘Task Listing Program . “. a ae ae 

. -Task Validation and Selection. Program Care. 

.. Syllabus Development Program ”. = oe ae 9 

.. Training Support Requirements. Program _ ice ae 
Existing Materials Evaluation Prog Bram ac ee & 3) 

| Priority,  o, e 


0" program number referred “ii Figure 1 


ie “Taetics” Package Program; aa! os oe, : BBE 
2. Trainer Requirements Program an bP. ee Gh) Ree 
(3.--‘Problem Analysis Program : sgn ae ose Pee Teer 
ai Master Plan Program ~- ° 2 vy. 00 UU CRs CA 22e 
5.,. Procurement Package Program © PP ie ees ese Me Fae 
6. Mavy Training Plan Progrem:. ~ ae ee ee) es 
7. Procuremént of-.Trainers Program (© 2 ne 0 (20) 
8. Implementation Plan.. Program ee et ee ae 

9. Quality Control Program . ge OS og tee Ve) 

10. Historical Trainer Data Program r i (18) 


a Le ‘Existing VIS capabilities: ire cds address many FAISD > 
' tasks, nowever, ‘and therefore may somewhat revise priorities, |: 
since Boca are. availble at ee small, wl f any, additional | 


pias 1. shows one possible oopesenta con of ‘the set of. 


gale te Systems sdescribed in Section II, which would, when = 


linked to each other and using the -intezrated data’ ‘bases 
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_ 3 > [Process Monitoring hoe elegcl Hee Since eel Trak: og :. oe 
nee ae oe "Ml Actities Show) | ne ae, 
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“The priority of program developnqe Snecested by el 
ore FAISD task- -to-program relationships : indicated in 


| Figune 2. indicates’ the following. preliminary: program SAlao ae a 


priorities: Notice that many of the FAISD tasks. would. be. ‘Served. 


by: the .same. programs (i.e. 25,1,..20.2, 19.1 on. Figure 2) using .¢ 


‘different. data. The analysis. of-VIS capabilities shows that 


ee “HAVTRNEQUIPCEN 71-0-001k: 1 -. Ss. ee ae 


‘several of the programs . are either eurrently: ‘available. in Some. i 


“form; or are easily adapted. from. existing: programs, data, or 


“+ bagie systems capabilities with a minimum. of extra effort. se 


"vinelude’ the ‘student entry level, task listing, objectives 
. -< hierarchy.#-syllabus, development,” cand: ‘computer. mariaged . instruction: 


(programs. when these, are taken: into: BeCOuRE the: ead 
op Tighing.-. ‘becomes: ae oe oe ee: 
go re iy det ‘Priority 7 | oe . = ee - 
es program se Figure 2 eee Reference. Number 
_ Standard Capabilities: ae oe co 
ca Lesson, Specifi¢ation: ‘Program’. eo eee =e) 
- “Lesson: Authoring: Program. RS » 13)? 
_ pee, ‘Production Program: Cog ee . (14) 


:. Lesson Tryout Program oe ae ee ae CO) 
» CMI Program =... _ Eee aaa ain at C18) 
“Media Selection Program: eee ee, ee 
. -Prégress. Monitoring. Program ee ek ee 
. ' Student: Entry ‘Level. Program. : et ats ( 
0. Task Listing Program: ee ee ae | 
1. Objectives Hierarchy ee eacae Program ( 
2. aca rev ro OnEnt. Program ( 
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? CAT Linkage Specifications. Program rr © i an 
2. ‘Task Validation and-Selection ‘Program ~ © °°. (ut). 
«. Training» Support Requirements Program) ©. == .€ -9)° | 
Existing Materials Evaluation, Program. 3 2 0. ( $Y 955. 


Master: Plan Program“... -:. g (- Huse. | as, 
/Implementation Plan- Program ©" <t.. c.0° = Cs 
‘Historical Anetner Data Program a eee ve AB) 
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7. 
8 


. Program. | 
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oe Other. on-line aids’ to ISD. are under a ae ieuaene: within ” 
DOD. With one major exception, these are not yet complete or  , 
‘available. That exception. is:an Author. Management System CAMS) 
completed under -DARPA Contract #MDA .903-76-C-0216.. : The . AMS is | : 
- equivalent to the Lesson Production Program, already: a. Ys (2 ot 
_ » program, so the. :prforities. do. not change- However, since the... 
‘tcompletion , availability... ‘and applicabil ity of these. other. : 

. efforts. ‘could be very. important. to. CATSDM -in terms of changing. 
priorities. 3] and "3, 20 tasks. 5-0, 5.4, 5.2; cand 5.3,0f the ._ 
_: basking | dnd- phasing. chart. in Figure. 3.5 4b is. ‘suggested that 
these. efforts ‘be monitored throughout. the. CATSDM opine sige 


The ‘tasking ‘Laia: out: in the. eestine ees hart. shoud | 
represents the . study. gout is s best. _judgement of 2 reasonable. ~ 

> time :-frame in. which.to - accomplish, ‘ach: task shown. * It-is._ 

“understood. ‘that the. hardware upon’ which CATSDM “is - 

implemented, “the ‘software facilities available for cATSDM 

-. development on“that- system, and the level of detail, 

. sophistication, . ‘and complexity- finally. décided upon. for. the 
‘emerging. CATSDM: system, ail will. ‘have van. impact yupon:the — 
“Level of effart. required. Tt is felt, ‘however, “that. within 7 a 
” reasonable Limits; these degrees: con ae variability ‘Can 2) : ee 
“accommodated in terms. Of increasing: or’ decreasing’ staff aad pe 

stale ‘be ‘accomplished’ ‘within the ‘approximate time frames_ oe ee eae 
Shown. Notice that the tasking is set-up in. terms*of a. “0.0 Uo 
three: year. initial development program and, of course, any. 
ongoing maintenance revision, and enhancement which would be 
-desirable would continue ‘beyond that period. .It is the — 

_ feeling. of the. Study, group that in -this implementation. 
schedule, the end. Of the. Tirst: year would show maximum | 
payoff, in’ that- the highest priority functions: and .-: | 

. capabilities- would then be available for ‘use. 


a 

. In this suggested . -implémentation. ‘plan: three major aréds. 4 
OF: Aevstopment have been selected for preliminary - | | 
; implementation. Tt. is. felt that, a — standard. 


ote . “0 @ 
. s 


‘Author: Management. Syste a” ands CMT: systems - are ‘enhaficed- and 
2 es by development of:. the selected. Priority. a pxOgrans, Se 
Suggested in this document, the major part of ‘the CATSDM a 8 Nae 
., system ean be provided’ by: the end .of:. the first. ‘phase. - oe ddan 24 
Again, it should be-realized thet~«the number ones priority. omer a 
- progr ams attempted. within this.context; the’Level-of de tail et 
and sophistication aspired to on ‘this first. -cut, and other. ge ee 
-flaetors all. will- influence ; at the very Least, the ‘level co a 
magnower Which must bé:.appl ¥ed. to: -the task; and: potentially ~ © 
P ERICA obvigusly, significently, impact the time phasing- of. 
7s lementtation plan. -It should be realized that the 
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-roup pel teves gould: provide. 2 aes) high. Gaye ‘of Isp. 
_ Support service, but perHaps. without all of. the "bells and 
whistles" that.the eventual, mature, CATSDM system would 
exhibit. It is sfurther- felt that? a program/system. . 
“specification, design, and development. evel. of effort 
* reasonable to. accomplish the. Priority. 1: ‘programs in. the 
firse year, as shown-.on this-tasking. ‘phasing - ‘chart, ‘sould 
reasonably. be “accomplished with. about: ten ual aed of 
.ef$fort. | 2 os 
etn the following: ‘explanation of es eaee, ae ‘showld re a 
under sfood that azhe jindividual | priority. one programs ‘donot. 7 
Cheamy: fall Singly. into the basic: Capabilities AMS of CMI 
subsystems, in mast.cases, but rather ‘they distribute across 
those and..would: primarily be considered as additions, eg ey 
modifications, or. enhancements. to those systems. It. is. also’ | 
felt that in the case. ofall: three areas of concern, — : 
“existing capabilities provide; a: good head start on. most. ‘of 
the > -Priority:. 1 programs. - Otherwise. ‘their - design’ ee 
"development and- implementation, ‘in the time frames and- 
‘withim the Level of efifort- puta would cleanly be 
Somewhat ‘anrgasqnabled <2.” oe 4 
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—TEsk te ae ‘Rey isé/intorporate: Scheduled vts. expansion: into 
work > plan.:- It is.iclear, that. in the.event CATSDM is- a 

- implemented upon VTS thati the CATSDM development prog 46 
itself will. constitute a non trivial perturbation. rogpan the | 
cexisting. VTS expansion wand development sschedule.: »Thernefore, 
-it.is felt ‘important. £ an accommodation be reached aS xg. F 
early as posSible between these potentially conflicting ; e 
activities. ;Possiblecoutcomes of this . planning activity.. “«t... 26, 
would bea revisiog of: this work plan. ror -‘CATSDM, or er a 
“revision of. the: nee oe ances or aon oe Ss 


= ‘os: ae . 
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“ alee Pod arte” ai a ae os noe -&: 
- . * . * be 


rade NTS. Cipitcdrporate, garsom se: capabilities. ae. 


2, 7 Devel: detailed functional. Sf seeing ree standatd. —? 


Paw e ; Bag al ale Ata ee . “ he ag oe 
: 


capabilit: es. In this -tdsk the. general functional. oe or 


specifications: ‘listed for a CATSDM system -would: be: ekplaing# oo 
‘in more detatl and enhanced: as’ necessary, taking Into." 4.4 «> 
-consideration any: built in advantages’ or disadvantages: Of. oe 
. special features of ‘the particular hardware/ software sen 
- Selected.:.~ es oes a Sg eee 

Sone ‘oe: 7 e 7 re i. eae fe 


ie oN aren a Br : . ny ia a wee et oh at as he a 
: ‘a ‘ St Mp gs 
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2,2 ‘Nevelop: systems/subsystans. spees’ for" Standard 2.” 7 


fa 


capabilities. —. At this time it will be "possible tobegin tsi 

“ Aeyeloping. the ‘systiem/sub system speci i fications: for: the |. . . 
“ERICdard. Capabilities in any. cases where new. software. . -_ 
“GevElopment or existing: ‘Spftware modification ts “galled. for. 


Bo COS ae : 
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8-23 3 Develop > program’ specifications for eanaeed eee 
/ capabilities. As the, Systems/subsystems, specifications » » ae 
‘become better, development. oF the encarta ee can. 
Ape. ‘carried Ce ae ee ee ee ; oa 
a 2. 4 Code’ ‘standard eepanidity programs. In all eases. where 
standard capabilities are not already.existing on the © . ee 
a ae system, but are possible. to. program into the- system being © : 
. used for CATSDM development, coding’ can begin as:'soon-as the . 
‘standarwigapability pee specifications are well. - oo 
developed. . . ce a? 


“2.5 Debug Standard. capability. programs ., ve reeds of ugha 

_individual programs'involved is complete, individual debug 
oo of those programs.can.proceéd. As: multiple programs are... 

finished, syste debug can commence in ‘terms of debugging f 

‘the. interaction. of those programs with’ one another and the. 
1 existing hardwaré/soft ware. eae 


2. 6 Test taniaes ‘capability’ oroevams. ren: the various: 
‘levels of debug are complete. Or: well. advanced, testing of. = .? 
. the er eueere capabilities can. commente. : oe et Oa 
ee 
a 2. 7 Revision of st ndard capability programs. and- Se 
documentation. Efom the testing. activity, revisions or «=~ 
additions mayb indicated, these.can then be programmed -and 
final document tion of the standard “capabilities can- be~ 
eompreests - i. ae ee get : ge ca teae ate 


3:8. fonvert/expand Author: Management ‘System for use on VIS: = 
ee 1" ‘SuFvey: feetd- £0. identity vadditional goceat or necessary i 
"modifications to the: Author: Management. System. Thee. Author. 
, “Management System exists in the form of an Operating ‘ a 
prototype with extensive documentation. ,ROwever, by ene ee 3 
eS _, time rt. 2S. incorproated asa major program function ‘in the 
“. * CATSDM system,:it maybe -p ssible.to identify,.jin the “Tsp. 
wet i Plield, ‘desirable additions:cr. modifications ae could be": 
a “incorporated at this. time. : ee he ees 
ee ; ~ A. . . 7 nd ae : " f- ; ; . 
3. 2 “An lyze AMS requirements. iirespéct. to VTS. capabilities. *> 
a criti¢al part of accommodating - an- existing. ‘operational - * = 
system ‘such. as the AMS onto a different. hardware/software. 
ee system TS-an- analysis of the target systems’. capabilities,.: 
rae in respect, to ‘the requir. lent - {either-impkied or. explicit: 
‘ ~ of’ the system. being translated. Te ‘Hs* felt” that eee ‘can “bd 
p= Cpa in a fairly short | oe 
FLA Fut Txt Provided by EC 
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a on .; Develop: eon ‘Pkdtibne al. ee ne ee for” ened 


Oe ae : yo  * oes oA ae Ss 
- ee NAVTRAEQUI PCEN 77-C=0018-1. 
30 avevelog ‘détailed- ‘fune etional specitications tor “expanaiae 
AMS on VIS. - In. those cases where AMS is to be expanded.- as a 
result.of. the. survey of the. field with new functions’ or | 
capabilities, ‘detailed fiinctional Specifications will be wo aa 
evens tor: _ these enhancements, . Z fa, » 
iS Develap: system/subsystems specifications. fOr: “AMS 55 | 
ee the. detailéd functional specifications, ~ ' 3 
systems/subsystems specifications for the AMS, for the os 
“modifications to. AMS, _and for. the enhancements: re AMS, will. 
‘all be prepared. r , * 
"3.6: Develop program Sees ieakion. for AMS. When 
Vi ay aten/subsyetens. specifications are complete or well’ 
_advanced, prea specifications can. begin. 
-3. 7 Code: AMS programs. ° when program “specifications are 
complete or well advanced, “coding of tie snd ividuel 27s 
PEOeT ans. can commence. aa a 


a 
, 


| a Debug. ‘AMS prograus. - AS" coding a ee programs is = 
completed, debug’ of. those. programs, both eine urerdy end 16 
» ¢onjunetion with one ee. ee Pe started. fee |. chad Soe 
-9 Test\AMS- programs. and ‘systems. As hehgee tne: ane ? 
fhaiviaual: programs is complete, testing of the AMS as a “ 
‘system operat g.on the host, eg dne Soe Nene system can- 


. begin. 


a ¢ 


“o 10. Revise: AMS” ‘programs: and ‘documentation « Qn pee, basis © 


of the. testing. results, revisions to programs may be _ 8 
Sec get act documentation of the final AMS syeten will be > 


q 


Gomes et o 
Sesigeaet ta de aint 
2 0 Revi é/update/expand VTS. CMI capabilities. 
we, 3 ‘ 4 
al Saves ‘of: CMI. capabilities and requirements. A survey 
arr be’ conducted ‘of existing. CMI applications, beth within 
and. without the military ,.specifically in regard: to. the - 
“needs of the ‘potential: CATSDY: audience, and’. a report of “~ 4 
~ desirable CMI characteristics discovered -or identified will case 
“be prepared. -s BT eee © eee, 2 ee. a Fe a : os 
8, 2 Analyze VTS CMI. In, eet eas ‘Simul tanéously, ae existing. 
OMT. ‘capabilities: on: VTS: will be. pan in. some detail, and. 
"10. -conjunction with the: tsUrvey, . — WLLL be made On - 
“ncaeats “characteristic in. te rms of its availability © _ 
Sige eos ow NTS ~s a a a ee ee a ee 
ia ae a ss 2 ie - fae . eS i" gee Pe ee ee 


Cee 7 NAVTRAEQUIPCEN 17 -C- e018-1 
functional ‘specifications. for ‘those additions’ on row 
enhancements will. be prepared and incorporated. into the >. 
functional - ieee, ak the> existing: CMI” system. - 


ee 


— uly Develop new or’ revise existing. sysStems/subsystems a 


specifications... Similarly, enhancements or revisions. ‘to-. the 


<] 


systems/subsystems ‘specifications will bE pvepered. as 
Heese seh ys ac | —— a 


- + ce -: 


“BS Deyelop/revise program:-specifications., The: program as 
» Specifications will be either gsnerated Or reviséd, and‘ 
one aay aces aE the. existing program Specifications. 2 4 


4.6 Code CMT. programs. ‘for VTS. _ With the -comp @tidn or>. 
advanced development of the. program specific tions, ecding. 
can begin on CMI - ee ; 


ea 4, 7 Debug. CMI. -programs for VTS. ‘The additi ns offrevisions | 


, can, be. ‘debugged as coding advances « 


can begin test’ and evaluation: as: “debugging is complete. 
4, "Revise. CMI Programs--and ‘Documentation. for VTS«: As 
ee eee advances, programs can ‘be revised as necessary - “and: 
“*F the documentation ll the CMI -subsystems.. “can Se UneGatee 
appropriately »* a ee a 2 er rn a ane oe a a 


- 


3% fe} Update ineopnetion eoncenning progress. OF applicabie., 
“CATSDM. functions .- Ts 2 tee ; a 


od 
’ 


5. 1 ean ones telephone surveys. _ Cn 2 “periodic basis — 

throughout the CATSDM development. activity... information 
‘néeds to be gathered. and updated concerring other: DOD . 
-, efforts. which could be “adapted or. incorporated within the 


, CATSDM system. -One part.of- ‘this information gathering ~§ as 


. process’ will be telephone ‘surveys to other DOD activities... 


5.2-Review: technicel. reports, progress reports gna Sther: 
‘information... ‘Augmenting’ the telephone «surveys | will be 


om, 2: Test’ “OMI Programs for VIS The ‘revised. VTS CMI--program 


c 


literaturegreviews ‘and review sof such documentation one other. . 


activities’ as may be: available: ae 


= Ly 


" 5.2 Major project briefings/demonstrations/planding 
~“ ¢onferences.. Towards the end of each major phase ef the 
“CATSDM, development, a major -project, briefing should: be held. 
““in whiéh a demonstration™ ‘of work complete®m within - the phase. 
x.can: be given in:terms of actial, exercise of the funetions. 
_7 «This. Should take place in the ‘context. of a planning i. 
ee conferenae . in. which an evaluation* of the. progress by. the 


. for” rae next majér phase of the project. 


Pd © i y eos, , *e . ms ‘ aA a 
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planning. group: would. generate a reviséd tasking ane. Pee 20S 
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“6.0 Re- evaluation: ‘and reordering of. prior ities: for rssaram | 
-development. ~ On “‘the-basis of the informaLion ‘avail: ata 

‘concerning: the progress made-in each: “phase ofethe = 7660, 8° 

and upon the information available concerning. othe: oe , : 

~aetivities which.may be applicable, a revised list or. 
“priorities. for program development in the next maser Phase. 
Of the» project. will be ‘issued. — Dion ; 


: _ ie 
a 0 Develop. ‘Second priority Tevet ‘progrems, On the | besis.of “= 
the reordered priorities, a’ work plan ‘for the second phase 
will be developed eénd,..as appropriate,. work.may_ begin one the, 
development of the- second -PriCEtey. . ‘lev I PCR ea aes ea TS 
> a : ag oe ee a ; ‘ 
2 0 Develop” third Gases programs.’ tollouihe.a: second eg * Oe 
“phase major project briefing/demonstration/planming a ; 


z 


- 


‘conference and a re-evaluation and reordering of priorities ih Seg 
for- program: ‘development, 2 PESANSHE and work for the third 2% aes 
phase will Proceed. . 2 cua! a oe ee 


DEVELOPMENT AND IMPLEMENTATION “RESOURCE REQUIREMENTS - 


Several. factors wilt have\ considerable effect on- the. i 
mesources required to develop and~ implement ia CATS DM. system. 
One is the extent.to which the system base from Which it is, - 
to be evolved already meets many CATSDM needs... VTS, “the ss ; or 
most attractive: candidate; will already suppor,s” cS range “Of *. ee a 
CATSDM needs.” However, the VTS is itself an i@ierging, oe oe ae 

system, and the status of documentation: issuch ‘that “the es nee 
full extent to which the VIS meets gee SDM. tfeed s’ is net a ee aa ee 
precisely eters : RTI | a ak rn a nn 


a a er 


“Awother Posean. is the av ailagieey St ‘biaer oS Te ROE ake 
CATSDM-related. functions ;.d¢veloped“@ithin DOD,. which. conti : i 
be adopted or adapted, @s well ag theextent te Haren @ se Gee, 
existing ‘TS functions’ will -rregd to .be adapté o ;be oseds a - 
Obviously some, such as the | Author *Management_ System: CAMS) 

deve¥gped for DARPA, are prime @andidates for’ early re a en 
attention. Other's. will haves be studied before-a-precise. 2 2 
estimate of Fesources required. to make them isefer* could a 
made. 8 ee eae — ey 4 ns To beige VE ee ae 

: > : ‘ “Bs at gig AG ee =. naar ae ; . - 3 

— “Another factor of fecking: needed. resource’ oct ie: the ye : 
extent to which CATSDM speciftéations @ould be* incorporated: a : 
zinto. ‘future development. work, already. ‘scheduled: (such.as the. eee 
continuing work “on. VTS) without significantly: increasing «f° °" >." >* 


ae GN g This, ‘ee: cannot .bé known at, TS REL ee Pe 
#. ons : - a 2. 7 nee 


. 
e 


F) 

“3 And ‘finally, this “élass, ‘of. problem, that of’ estimating — 
odes and resources required fora major, complex, - _. oe” 
“sophisti¢ated ; information managemént, system. for | use by a oe. ae 
wide, rangél of’ personnel, is essentially open= ended. “That . ~% 263 
18%, op ime a itd ma ¥ SE some. useful’ minimum threshold Pera a“ 


2 *y . =e a 7 ' 7 “, ~ : . ‘ S : me i ae % o - : 7 a : va = ; ; - = am - : a : d 
7 e ae : a | : Sets e . < tae “< -) ae rt _ ~ 7 - i . = ie ; : : . es ‘i . : # ae a 
29 eae OT eee ene, 


es ee a an ; : eo ae ene set 228 RSS Rae Be Se 
e > oo 3 2 = oe . a er. a re t we is ae 
: r : ae = - . te we ie ae nen 


a4, 
> 


ae = Ce? «., “NAVTRAEQUIPCEN 17> c-0013- anne | or 


zn j whieh a ‘useful - ‘eysten is suse not renee Lt #1 possible. 
ae to -add refinements, sophistication, and human engineering 
eee considerations: which. would. .in’ some environments be. very: 
: “important, -and which ‘would - effectively increase costs 
™ a “ventoid over. that Shreshold ae 
ve * : a : . 2 
ae i oa ‘HoOwever,. based ‘on ‘the assumptions Visteéd “under: the. 

—_ previous g@ ection, upon -the experience ofthis author with® 
es developing a major. sATSDM: Subsystem. under. DOL funding ~ 8 
a already ,. and “upon the high level. of expertise ‘Of the. VTS.. 
staff and relative utility of VTS"as it is currently 
understood, “the. following eeu iiereo care oP igree with. some. 

confidence. ge ee ee ee ae nee eas. oe 


Hardwaré/Software Resources a Se fe He gee 
ae a ese Pid ' - 
The. ‘VTS, as* is, is. potentially 2 ees sendece CATSDM 
: enor, with a few reservations. One is storage . 
7: Capacity..*To provide the -on—line: curriculum maintenance 
-services so important. to a‘ CATSDM system, ‘much more disc: 
anes “storage is needed. Fortunately the technology has ‘advanced 
Seo” papiddy. > For ‘example,. on today's TICCIT systems (another © -¥. 
ee as minicomputer-based: system) over. 300 million bytes- -of storage 
a4 ean Ye provided for. about- the same price “as 25 millién bytes. 
. only six years. ago.. ‘If. VTS-is used, about this level . a 
on additional, storage. (300: ‘million bytes) is recommended. PEOE SE 5 
‘ aoe cadded | less than 50, as per SYSE eM. ar 


~ Pod 


- (However ,.if one. 300M drive is. used. ‘instead. of: the. 


2 eo dra vos currently. installed ‘with VIS; the. savings might: 
no peer ven! CE oa ee © ae ae ie oe ee 
ke One: ‘ether item ‘is peetens snamely advariced Gerda ud 


ee, -maintenance capabilities sutn: as:.a.system to. ‘Allow-entry of. 
ee eee large amounts of curriculum materisls: on-line, the ability. 
1s to search, revise, and: edit this data base, and in géneral 
- to: ‘manipulate: ‘it “in a Variety. of -“ways.. “Commercially: such 
,capabilities. could. ‘be -provide®--for | under Sie, 20 eer a 


(Or cexisting: VIS_ ‘capabilities could be- enhanced yin -nouse, ne, 
Sean NWOLY Ss aoe ee ee ce oe hes 
een or oe a ee ee 
ee Progres, Develobmené Costs. cing o pee og ee 


OTHe Army Research Institute is “currently. issuing: an 
- series. of small (under. $100,000) contracts. intended-to’ - 


ae ar ‘provide: a: Variety of., products in the | area of computer “aids 
Toe “to ISD. This. “program will) iad = Avene: much, Less) ERen a- 
jovi + milton ‘dollers. . ee a pee : 


oe 


Se ee ee | EP 
Se a The “AutHor: Matiagement: ‘Systen’ CAMS) - eee ened. under DARPA. 


te J Gon tract. ‘was “conceived, designed, developed, vand. implemented. 
pee for under. ‘$100, o00, including hardware. PE ie teres Sane iae oe 
2 COs a be ee ane a 


_serateh, it. is estimated that. a comprehensive CATS DM system,” 


’ 
at 


a aoe Re om Se 7 a: ~ * Caer oy ae ee ar 
o ea ~ a a oo 8 i ae aaa 
oT ae -. os f Biase 


= NAVTRAEQUI PCEN TI ec+ 00% at" : 
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compar ine the scope” of stiewe “two cATSDM. related efforts — 


be the. ‘whole CATSDM | system, and assuming: mutch will be uSable 
from these. efforts: ‘at -a-‘savings over implementing them -from 


based on -an enhanced VTS; can be fully implemented within+ 


NAVAIR .for under’ itwo million aotlarey ee ME 


n 


e 


hardware. 


Depending “on. the: acceptable minimum ‘ete stoid: of: gystem. 
sophistication, the degree to. which: the VTS, as it’ ‘is, is” 


-directly applicable ‘to CATSDM, -and- depending - upon the 7 
soe easily. adaptable functions from ‘other. DOD sources, the 


peost might be-less than one million ‘dollars. ‘However, | 
ardware costs.will prevent this from decreasing much-more, 

manless the standard VTS hardware mix is upgraded to take 

advantage of the advancing, technology, in which case 


~additional . add-on. Ahardware/system capability will aot be 
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SECTION Y ad ca. | cee 


CONCLUSIONS AND RECOMMENDATIONS | 


Stee This section: ‘of: “the - report will. attempt: to. ‘summarize 
“referencing previous ‘data. items as necessary). ‘the findings- © 


“of the ‘project: ‘study”~ group - in ‘reference to ‘the need for 
~ CATSDM:. Support ‘and. the current and potential | provision of 


“such Support: by an 
Wilt also’ be outlin 
for" implementing. su 


exis 
ed, 
chs 


"assumptions, and summer 


CONCLUSIONS 


ting system (VTS). Some discussion 
on @ general level , Of alternatives: 


upport based on asrange of reasonable 


y recommendations will be meade. ° 


The ‘analysis. of “CATSDM requirements and VTS. proceedad 


eyeter et across a series of data items. Some 
conclusions. are appropriate from each end will be ‘enumerated 


individually... En addition, | VTS analfsis conélusions auc the 
(overalt conclusions of the “study will Pee iver 


Data’ item, Q001-List of Candidate ISD Tasks 


os The” specific tacks chin ena Plest: kytat ion ISD (FAISD)- 
“modél can: be Classified as, to the kinds -Oof automated Suppor 


which would ‘be appropriate “for each: “Seven generel 
categories of oe ae Support - were referenced in this sees 


(classification: 


“On-line eo 
-On-Lline’® Support 

: Maintenance 

.- . On-line Support 
On-line Support 
*. On-line Support 


oe It was ‘conoluded. that. automated ‘Support would be . helpful | 


Be On-line. Support. 
-%— °° Cn- line Sigper t- 


Pon: 
for 
“for 
for,’ 


for 
for 


to 


fOr. 


Data. Collection - 
Text Editing/Curriculum Filey 
Sehéduling mee = 
Standardized Reports. 
Simulation/Modeling - 
Statistical Analysis » 


pate Searches and “Reports: ” 


“in. every one of the 64 SpecificsFATSD task’ area®. Every. 

task. was appropriate ‘for at least four‘of the-sev.en 
“eategoriss of: potential SUPpOort and the great. ‘majority. would 
.. be” helped in (Six to Seven: of une categories Oe potential. 


enue ron 


Data Item 9902 “Process Analysis ‘of “aidsp 


” 


ufficiently ‘well defined in. an algorithmic (flowchart) — 


face Tre great majority of. FAISD ) specific. tasks ‘could. ‘be. 
“2 


ashion, and- ‘to. s‘level, of | detail - @s to indicate. ‘that. the 


ee ", potential - benefits. from ‘automated aoRPort would a 


ee oa eg 
oe 


ih ig te ‘Gud: 
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‘ 


Data Them: 0003, -  Cost/Manpower Analysis of the ISD Peocess. 
fe s yes a ‘ 

mee er Data. Coltection item only, 

“eonelusions a eoeny ee for Data” Item Oo04. 


2. 


babe Item: ‘2004. ~ ISD: Task Pridrities for: cATSDM ae es 


On ae yao 43 packers: based on ie. etependent o WA 
décisions for. each task) of rating points, ,it is possible “to 
.rank the specific tasks of the FAISD model in terms of the. 
desirability/cost benefits to be achieved ‘from automating 
most or part of-that task. | Twenty-eight. priority levels 
- were ‘identified ranging from a high OF AT priority: points © to. 
2 low of T4. . so 3 


L eT ° ' ¥ = * iM _* os 


fou 


ae is possible to p r.sue Benepe on of /CATSDM ‘functions : 
“on a task-by-task basic -g.-, STC's Lesson Materiai’~:- ; 
Authoring and _Formative tryout System, programs LAP, LPP, 

wand LTP, respectively) or on a capability-by- ~capability. 
-basis. Ce. &-,; Review/Comment. capabilities). An overall cost: 
benefit will be realized, however, if the-high: priority. 
Subsystems are developed first on hardware/sofkware systems a 
providing. potential support in all ‘Seven basic~ Sapper e: 7 
cieteeerice identified in Cata Item 9001. 


os the mos.- cost effective subsystems are devetened: : 
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Costs Over time. . ; ae a . 
Evaluation/Revision table of types and amounts of - 
Manpower manpower required for ongoing 
Requireinents evaluation/revision 
he i \ , 
Evaluation/Revision of all costs associated with a 2 
irea ANaluation/cevisloa of program . 
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- LESSON SPECIFICATION PROGRAM (LSP) ee : 
% 5 . » . : 7 : . iG . . 
. = eve ee rem 
: Data Comments 
Coune, Unit, Lesson ULS a mulidigi number separated | LSDB * could be automatically input aid displayed asa cue | . 
number | ae by periods rea $DDB _¢ mena choice for ue. ; 
Lesion Tite LSD, elute dntiseincl | opli "could be antodtatclly lap and played aac, | 
~~ os ‘Nessoatopic _ | from SDDB } or seu oholoe for user. ¢ . 
Leuon Objective = «s P LOUS pou dain of coda spe - supptiod | program will supply objectives as it appears in SPDB - 
ts "| behaviots and standards " |Yeom'SDDB | and user can verily or modily 
Lesson Introdyction prose description of information — LSDB ss 
7 , ; oe to be presented as introduction * i ey . : 
Lesson Testing Logic is po ei of LSDB | incractive | uses wil pgcly data such at wheibar tess lo mastery | 
eis a Gaveloped i. i i or diagnostic number of items, mea evalua. 
Fe tion criteria, et, 
Course, Unit, Lesson SD‘ | amubi-digit numberseparated =| LSDB - ied. | could be automatically put and displayed as a cue ‘ 
Segment number . - | by'Periods ~ \ oo SDDB| or menu cholcefor ust | w. " 
‘ ; a . \ ; ‘ ; 
Segment Title — - brie! prdse description of phliod , 4 
i 7 . segment topic, « / ~ 3 a) 
: ee 4 . 
Segment Odjective(s) supplied '|" program will supply objective as it appears ia SDDB 
. a tdchaviors and standards ok from SDDB Stee on reiy cently ote. ; 
Segahont Medium >“) pumecical code indicating = plied - . =. 
‘, tedium _ 
ge. : . . ® ; 
Generality - 4 +4 1°. prose of varlable length =. interactive | core information as it is td.be presented to students “)* 
. | ’ : a . = | ; : : so : : : 7 . . ; u 
Generality Help HLS | prose of variable length \ | interacti spporig information to be presedited to students 
, 3. . : _ i ‘ a 
. ™~ oh , \ 7 au , 
Example Specification —_-|] EXSLS : tables”: interactive defiptioo of type and scatacal examples 
af! , oe . . 
Practice/Test "description of type and number of examples of 
Specifications - , . - }beanching | practice and test items, / ‘ 
/ . A - i . he ‘ p, F , . 
i/ ~ ' 
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Special Teaching 
Points 
Common Exrox 
Sraphics 
Specification: 


Source Reference 


Objective Classification . 


| prosa‘of variable lengths; tables 
(ae aaa 
; prose of variable lengths; tables : 


prose of variable lengths 
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"LESSON SPECIFICATJON PROGRAM {LSP} 


description of any special teachlog points 
description of common student errors, related 
guidelines for student feedback =. 


description/special Lastruction for ervation of ant, 

Photographs, ec, . _ 
Gascriptioa of source materials to be'used as references 
in developing thelesson ” 


classification of objective by predefined taxonomy 
(such as wselrule atc.) 
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IMPLEMENTATION PLAN PROGRAM (IPP), ae : 


: . Data ; os 
or Method ). a ae 
Description of ee of Entry : Conunouts 
*  Pbciel descriptions of instruc- ; 
thooal system atructureand = ws 
characteristics ; ; a 


~ fos 
Shedule of classes, with start ive | prograro will ask for student throughput data and 
ond dates and required schedule facilities, equipment and personnel resources 
as specified in TSRDB, Attentiog will be called to 
resource inadequacies Kdentified. 


Facilities Utilization j ilities and thei di ive | provides for specification of space for storage, offices, . g 
Dara . ing * | \earning centers, traines facilities, otc, [AW the TSRDB | . - ¢ 
a ‘ 
Equipment Maintenance iP ist of equipment and mainten- interactive | queries user for maintenance and wtilization data for a 
| Utilization Data : ance and utilization procedures i equipment specified in TSRDB °° * > 
Instructional Materials ‘ ist of materials and provisions i ive .| Queries user for replacement and storage provisions for m 
Management Data for replacement of consdmmal ; ing | materials as specified SDDB, + e 
a, items. a oe ie. % 3 . 
Student Managemen «| SMD. provisions for student entry o 
Data: proypacs monitoring, partici- aching - . So, 0 
Tow. patiga in training s oe ; u 
Records and Reports list of'required recordkeeping , interactive | program will make suggestions based on data from e 
and reporting procedures ‘ ing. | logic concerning service wide repdeting requiraments. SJ 
. . sg » , ' 
provisions for training, profi- 0 
ficiency evaluation, etc, of staff i . , 
@ , 
4 { a of ; 
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Formative Evaluation 
Roles and Responsibilities 


Formative Evaluation 
Evens 20 lic‘‘ 


Small Geoup Tryouts 


Formative Evaluation 
Variables and Method) 
of Measurement 


Internal Quality 
Control Roles and . 
Responsibilities 


lnsernal Quality 

Control Events 

internal Quality : 
Control Variables and 
Methods of Measurement 


External Quality 

Control Roles and 

Responsibilities = 

Exteinal Qulity 

Control Events 

Exremal Quality 

Control Variables and 

Methods of Measurement 
fA * 

Quality Control Data 


— 
Analysis of Data - 


. QO. 
ERIC 


tables, charts, reors ‘escriptions, 
codeg 
| flowcharts, piv ...pions 


flowcharts, tables, prose 1QCDB 
descriptions \ * 
flowcharts, proce descriptions QCDB 
atc, 
tables, charts, prose QCD 
descriptios, etc, ea 
. . 4 . 
flowebacts, prose descriptions, | QCDB’ 
atc, 
” | owcharts, prose descriptions, | }QCDB 
cr 
QcpB 
QCDB 
flowcharts, proudeipis, Cin 
ac, 
ouspercal des QCDB ” 
| sais programs Qcpg 


le 


Sill description of pecronnaljayolved 
branching | their roles and responsibilities 


form driven | descriptions of procedures to be followed, sequances 
of events, review aie a 


interactive organizing small group uryouts and pone 
branching aneere. rare to LAP and LPP 


form driven “description ‘of variables to be measured; of mathods 
to use in collecting, organizing and i intecpreting aH," : 
forins to ba used, atc, 


intesactive | description of ee involved, their roles and 
branching responsibiltie 


form driven } descriptions af sence to be followed, sequenced 


uf events, management controls, revision points. 
fores drivea | deo. iption of varlablas to be measured; of mathods 
wo use in collecting, onion laccpng da, 
forms to be used; etc, . 
interactive description of sta or their roles and 
branching cesponsibilides 


fom drivec. ; description of eveots, procedures, management of 

H ore revision and updating 

forma dian description of variables to be measured; of methods 
to use in collecting, organizing and interpreting data, 
forms to be used, etc, 4 


‘interactive | raw data which will be analyzed; data referenced to 
branching! | OHRN and CULS ; 
batch i 


batch eniry ‘programs ore sort, do item counts and ig 
analyses of data“collected. 


hoe, . 
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Course, Urut, Lewon, 
Segment Number 


Lesson Specification - 
Components 
Segment Examples 


Segment Practica/ 
Test lrems 


Review Comments 


Revision Specifications 


Story board/Script 
Lecture outline/aids 
Student Worksheets 


Tryout Procedures 
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3 LESSON AUTHORING PROGRAM (LAP) - 


Description of Data Comnvents 


multi-digi number separated 
by periods. 


—— OM 


“this includes LT, Ld, LTL, ST, LO, SO, G, GH, 
EXS, PTS, STP, CEA, GRS, SM, and SR 


created by author in accordance with EXS, CEA, 


examples to be used in 
STP, CRS, SM, and SR 


| segment 


"| peacticn and Lest iteras to be 
j used for segment, with appro- 
priate feedback 


genorated by reviewing authority 


prose comments 


generated by reviewing authorty or vother appro- 


prone statements indicating 
priate ‘person from LTDB. 


neoded changes In materials 


oe by author as required by SM and SO 


SS.LA formatted prose atements 


au by author as required by SM and SO 


LOA.LA —+| formatted prose statements 


formatted prose statements 


any special procedures regarding 
tryout and data collection 


SW.LA 


crated by author as required by SM and SO 


created by author as required by SM and SO 


TPLA 
: indudes necessary data collection forms 
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created by Ruthor in accordance with PTS and SO 
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Sours, Uait, Lasso0 ' coulti-digit number separated Pi 
Segment Nombéte by ee 
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roduction Tasks } 
roduction Staffing pessoal will be allocated for each taxkia PTA - 
a a : : 
1 mat in Codes manipulate texto create copy that is as 
oe ee 

_ ; . «| activities ; 
ypestting Code codes which translate copy i 

oo into pasteup o¢ camera ready 

‘ masters using phototypes- 
; setter... 
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LESSON TRYOUT PROGRAM (LTP) 2 é. . 
. at oe . ¢ 7 ae ; 
“Comments aa 
“ . ) rY ‘ 
moultidigt number separated ge a 
by periods. > 3 
listof dates when tryouts 
j occurred» . g 
» 7 ‘ : — , , . i x . ‘ e , 
Tryout Subjects ae names aod other pertinent, subjects selected in aocbrdance with target auditace Ay 
Demographic Data background data on subjects . characteristics in PADS and SEDB | So > 
Tryout Moaitor person responsible for con: . . /, ‘ ‘3 >. 
ducting tryout and collections ; oe ee *< 
hf ; “. data. y bi Fa ” ” — 
Trycut Procedures 1 incdudes necessary data collection forms, | Pa 
; ; _ ' : Q 
Tryout Test Data ' : o 
, » : ¢ \ : . QO 
Tryout Performance results of hands on perfor. ne 
Data mance testing with pass/fail : NJ 
, a 4 
Tryout Attivudinal _ | numeric data and representa: ™ ‘ 0 
Data * | tive prose comments from s e 
Revision Specifications prose statements indicating = 
‘ : needed changes in matesials 
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COMPUTER MANAGED INSTRUCTION PROGRAM) (CMIP) : 
of test format, instructions to be given 


ing’, studects and other cSnsistent portions of tests 


so ams fom wick specie canbe 
randomly selected — 


tp apt ‘ 


— ianals student: 


performance * | 
ae 


acives 


si cacti uae begun, . 
not bequa; projected competion, et. 


combinations of student perormanin data, pormative 


peeciel ers 
personne, facilities, muta pict 
tach soqment/lesson 
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Da 4 
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program defining learner control options, when learner 
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bas coatrol, program logic when proggam control 


. cat systems will wi extertul to CATSDM 
and linked to it from an interface for the 
mom part. Separation of logic and content, . 
potential for learner control, and Onting 
advice are desireable features, * 


infocmation to student On how to uw sy, bow well ‘ 
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ae HISTORICAL TRAINER DATA PROGRAM (HTDFT ae 


: > | Data 2 Metha y ¢ : 
Description of Data Ha 3 of En Comments ~ | 
nter 


HTDB »| form driven | RE ea 
PS detection, adaptability, otc, characteristics of symams . 


form driven srudsat performance data, ratibnak data ates" 


ioc orbe 


_ | formatted prose and coded form dciwa| structure and standardised ere 
data : ry 7 : : . ‘ i 
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much dig number separated - 
“| byperiods, : 


prose statecneat 


e . 


prose staternent 
list of prose 


ns 


Nactical Source 
ublications References 


Yojectives Hierarchy 
Relesence Number(s) 


mulidigit pumber represent 
ing related objective(s) 
. Unit, Lesion, ' 
as Number(s)» 
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narrative specification of 
Organizational resources that 


must be committed. 
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Detailed Specifications 


DIDPP 


CORPP. 


PECEP 
DPREP 


PEPPP 


EL PP 


-gpatractor,’ 


+ 


"PROCUREMENT PACKAGE PROGRAM (PPP) 


prose listing of all quired ISD 
activities by phase, « 


list of: reports, doconeats, 
ete, tobe delivered bythe ~ 


ie a 


format to beadheredtoby 


list of procedures to be used 
in-house to evaluate proposals. 


list of documents or otberdata | PPDB® 
to be included for amplification 


“| or clarification. 
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; avy TRAINING PLAN PROGRAM NT ae oe 
‘ av 
Navy Training Plan | dexcitia of background, Mary png | 
Introduction scope, purposa of plan; ” ’ 
Description of Nay hol nop eiptin a i, 5 
yun ee user can verfity and modify ee! / 
Ss =e . “ys. 
Contract Spa contract number, contractors involved, costing = 
Data. ; ‘norma, contactor ain, schedules,’ “2? 
ate | _ 2 
Dperation/Mainten- formatted prose narrative, — ‘Navy ia ii workloads, pe S 
mnce/Training Manning charts, labels . billet requirements, : x 
Trained personnel needad | rin penn nel party * 7 : 
Training Requirements sites, resources, ichedules, personnel needed to - Cc 
. suppor training acide oi ‘G 
’ a . e = eo: 
Major Decision/-  * description of major miestooe events and be m 
: decisions, command responsibl, date atc.” 2 
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“ 
, 9 
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_f | ° 
° 
: coi 
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++ PROGRESS MONITORING PROGRAM PHP). 
a . oe é . 2 ; 


’ 


: , | Data ; 
Description of Data Base. inpaed ry Comments 


Entered | 9! Entry 


a i : 
PMDB ‘| formariven| description of mart ‘as milestones, duadlines’ 
ideatificaticn of workdays, boldays, woekands, 


» | +, | etn of workin day, 
PDB — | form driven) porsnal, ~—-" ama 
' required stil, workload. 
“ Master list of fi : “| numeric codes and prose PMDB- | interactive | hind of skills the project vil ruin, tach ail 
| a af descipifn _—_ branching represented by ahusmerical code. 
- Maser list of events . svmaric codes, ro | PMDB . | formdriven} a bst of evants decisions, numeric i cade 
descripuons, etc. nc tions of event decisions; personnal involved 
7 equipment needed, tic. - 
7 ee eae eee : a) 
Status vector «= (tsi«t CV. codes and descriptive data. | PMDB reo whother an event in any biocarchy is begua, complet, 
ee Pe os BTW gs coir in progress and other related data 
Event hierarchies a: TER | flowcharts, codes PMDB | formdriven! procedural, ‘sequencial description of what events 
— ‘ } 2 ake plac wih any pa ot are 


Equipment) Facilities peose and numesical codes | PMDB — | form deiven equipment, facilities a for each event, code for - 
‘ ~ each workload. 

> aad < 
Project Accounts. 


prose and umeti description | PMDB | form deven sci ta bali coosderation, * 
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$- és ¢ 
rie, 94 a. 3 “GLOSSARY. a rn 
ey. ADOC. _ Analysis of a ee ‘Gontrols Seeereay ; 
of AMCE MS | “Aeceptable Media Choices-Code (Media. Selection Program) 
AMS?  ———- Author Management System 
C.So _, Calendar. 
CA.HT - Comparison Nigorithas . 
CAI ~. . .Computer-Assisted Instruction $e cigs 
~CAIDB - Computer-Assisted Instruction Data Base . ‘ 
“CAIP* Computer-Assisted Instruction Program: 
“CATSDM  . Computer-Aided Training System Development and | rd 
Management . a 
CD.PM . - Calendar Data’. . , eee 
CD.TP . . Contraints Data . : 
CDR.PP Contract Data Requirements : 
’GEA.LS - Common Error Analysis -. , 
CMD.HT°  . Cost and Maintenance Data aa 
‘CMIDB  ~ ~Computer-Managed Instruction Pata Base | 
CMIP , Computer-Managed Instruction Program 
‘CODASYL Conference on Data Systems Languages 
CO.TP. Classification of Cbjectives : 
CR.SD Criticality Ranking 
CSD.NT Contract Spec?fic Data . 
~CID.RE 4, Characteristics of Trainer. Devices 
CUL.SD ° Course, Unit, Lesson "Number : 
CULS.SD .Course, Unit, Lesson, Segment Humber 
_ DCR. TS. Design Cost Requirements - ; + a | 
DDP; CM Diegnostic Data and Prescriptions 
. DET.HT | Demonstrated Effectiveness of. Trainers \ 
_ DID.PP Data Item Descriptions» a 
DMR.TS- Design Manpower, Requirements 
DNS.NT Description of Navy System 
DOD: °"-' Department of. Defense 
DPD. MP Detailed Phase Descriptions . oop 
DS.PP -° -Detailed Specifications. e ° 


DTTD.SD © Description of Typical Training Day/ week ~ 

DVCR.TS Development Cost Requirements 

DVMR.TS- Development Manpower. Requirements” - 

ECVYM.QC External Quality Control Variables and ceo of 


. Measurement. 
EF.PM - Equipment/Facilities 
-EH.PM. Event Hierarchies 
EL.PP ~.. Exhibits List 
ELT.SD. Estimated Lesson Time 
EM Existing. Materials , = 
: -, EMED.EM Existing Materials Evaluation Data 
\\ EMEDB Existing Materials Evaluation Data Base 
EMEP  §—- Existing Materials Evaduation Program 
+ -EMMC.EM. Existing Materials Media Code >. 
1. v ‘is EMC.EM © Existing Materials Media Code = - -_ = . 
; ~EMPT EM Existing Materials | Presentation Time ae 
: 2 eee 2 ; ‘ 


BG dt 


-EMRN.EM | 


EMT.. EM 
EMV.IP 
EQCE.QC 


EQCR.OC . 


-ERC.TS 


ERM.TS. ° 


-ERS.PA 
EXS.LS 

-FAISD 

_FC.LP 
FEE.QC 


FERR.CC 


FEVM..QC 
FUD.IP 
te 
GH.LS 
GRS.LS 
HOOF .MS 
_HSC.TP 
HSRC.MS 
HTDDB 
HTDP | 
HTIC.TP 
ICR.TS 
ICVM.QC 


IME.CM 
IMM.IP 
EMR. TS 
IPDB 
IPE 


IQCE.QC” 


ITQCR.OC 
ISD 
ISDP 
_ISC.IP 
-. LADB 
LAP 
LI.LS 
LM.SD 
LO.LS 
LO.SD 
LOA.LA | 
LP, SD 
-. LPDB 
LPP. 
- LSDB. 
LSP 
LTDB 
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‘Existing Materials Reference Number. 


- 


Existing Materials Title: 
Equipment Maintenance/Utilization- Date 
External.Quality Control Events 


External Quality Control Roles and Responsibilities. 


Evaluation/Revision Man power Requirements 
Evaluation/Revision Manpower Requirements 


.Existing Resources Specification 
Exemple Specifications 


Fleet Aviation Instructional Systems Development 
Formatting .Codes 

Formative Evaluation Events: 

Formative Evaluation: Roles end. Responsibilities 
Formative. Evaluation Variables & Methods 
oe. Utilization Data : ne 
Generali : 

General ce ‘He lp 

Graphié Specifications 

Hands On Objective Flag. 


_Hardware/Software Characteristics: 
‘Hardware/Software Requirements Code 


Historical Trainer Data Data Base 
Historical Trainer Data Program 
Hands-On Time Requirement 
Implementation: Cost Requirements, 


‘Internal Quality Control. Variables and Methods of 


Measurement : 


_anstructional Materials Effectiveness , 


Instructional Materials Management Data 
Implementation Manpower Requirements 
Implementation Plan Data Base 
Implementation Plan Program - 
Internal OQuelity Control Events : 


Internal Quality Control Roles and Responsibilities 


Instructional Systems Development 
Instructional Strategy Diagnostic Profile 
Instructional System Overview ° 
Lesson Authoring Data Fase 

Lesson. Authoring Program a . 
Lesson Introduction - ne 


* Lesson Medium (Media) 8 


Lesson Objective 
Lesson Objective 
Leeture Outline/Aids 


Lesson. Production Data Base 
Lesson Production .Program 


‘Lesson Specification Data. Pase 


Lesson Specification Program 2 
Lesson. Tryout Date Rase Z © ss 
Lesson Testing Logic a 
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Lie 


- MDM .NT 


= 


MLE. PM 
MLS. PM 
MMD.TS —. 
MMPR.MS 
MPDB 
MPI.MP 
MPP 
MSDB 
MS P 
NTPDB 
NTPI.NT 
NIPP «+. 
OBS.OH 
OC.LS 
OCRTL.OH 
CCS.OH 


OBRARN.CH 


OHDB 

OHP. « 
OHRN.CH 
OMSD.OH 
OMIM .NT 
OPM.TS 
ORR.MP ” 
OSS.OH 


N 


> Ps PM 


JP Ps OP 


- PA.PM. 


PADB 
PAP, 
PCL.TP 
PCR.TS 
PEP.PP 


~PFESMP 


PGR.PA 
PGS.PA 
PHT.TP 
PJE.SE 
PMD& 
PMP 
PMR.TS 
POS.SD 


PPDB 


. PPP 


PQS 


PR.TP 


PS.MP 
PSC.LP 
PSO.MP 


Proportion of Hands-On Trdining Time. 


- as a 
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Lesson Tryout Progrem 
Major Decisions/Milestones 
Master List of Events 
Master List of Skills 
Media Mix Designator 

Media Mix Priority Rules’.- 
Master Plan Data. Base 
Master Plan Introduction - 


'-. Master:Plan-Program: 


Media Selection Data Rase- 
Media Selection Program - 
Navy Training Plan Data Base ; 


"Navy Training Plan Introduction 


Navy Training ‘Plan Pregram 
Objective Behavior Statements 
Objective Classification 


Objective Cross R:.ference to Task Listing 


Objective Conditions Statements “ 
Objective Hierarchies Alternate Reference 
Cbjectives Hierarchy Data Base i 
Objectives Hierarchy ‘Program 
Objective Hierarchies Reference Number 
Objective Media Selection’ Data ‘. 
Operation/Maintenance/Training Manning 
Cbjective Per Media 

Organization Roles and Respongibilities 
Objective Standerds Statements 

Personnel 

Project Accounts 2 

Problem Analysis Data Base 

Problem Analysis Program 

Performance Characteristics/Linits 
Production. Cost Requirements 

Proposal Evaluation’ Criteria 

Project Funding Estimate 


Program Goals Ranking 
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